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Executive  Summary 


This  Final  Technical  Report  is  submitted  in  accordance  with  the  provisions  of 
Contract  No.  DAAJ02-96-C-0042,  issued  by  the  Aviation  Applied  Technology 
Directorate,  U.S.  Army  Aviation  and  Troop  Command,  Fort  Eustis,  VA.  The  report 
describes  a  project  supported  by  a  Small  Business  Innovation  Research  (SBIR) 
Program  Phase  II  contract  (Topic  OSD  95-0021).  The  project  has  been  conducted  to 
explore  the  development  of  an  intelligent  symbology  management  system  for 
helmet-mounted  displays  in  rotorcraft. 

The  overall  objective  of  this  project  was  to  develop  and  evaluate  an  innovative 
and  intelligent  information  presentation  system  for  HMDs  in  military  helicopters 
The  subordinate  objectives  included  the  following: 

(a)  Define  the  specific  information  elements  that  should  be  presented  via  the 
HMD  to  support  pilot  tasks,  especially  as  aircraft  flight  regimes  and  tactical 
requirements  change  during  the  course  of  a  mission. 

(b)  Develop  a  powerful,  portable,  intelligent  symbology  management  HMD 
simulator  to  permit  rapid  and  effective  identification  and  evaluation  of 
candidate  intelligent  prioritization  and  filtering  of  flight  mode  information. 

(c)  Develop  and  test  innovative  flight  and  mission  information  symbology 
management  concepts  on  the  portable  simulator,  realistically  demonstrating 
the  most  promising  concepts  for  providing  timely  information  to  the  pilot. 

Section  1  provides  an  overview  of  the  background  and  general  approach  taken  in 
performing  this  project  and  describes  the  need  for  a  sophisticated,  low-cost  HMD 
research  simulator.  Section  2  describes  the  activities  undertaken  during  the  Phase  I 
SBIR  effort,  including  the  initial  information  requirements  analyses  and 
preliminary  interviews  with  Apache  subject-matter  experts. 

Section  3  examines  the  symbology  management  issues  of  mode  switching  and 
information  organization,  contrasting  the  current  pilot-operated  mode  switching 
methods  with  alternate  psychomotor  controls  and  comparing  the  symbology 
moding  strategies  of  the  AH-64  Apache,  RAH-66  Comanche,  and  the  ANVIS/HUD. 

Section  4  describes  the  heavy  burden  of  Army  mission  management  and 
communication  tasks  and  describes  HMD  solutions  to  the  current  shortcomings, 
focusing  on  the  concept  of  "augmented  reality"  as  the  key  to  workload  reduction 
and  situation  awareness  enhancement.  This  section  also  describes  the  mission 
information  requirements  analysis  and  the  124  information  elements  identified  as 
potentially  useful  on  an  HMD  display. 

Section  5  describes  the  design  and  development  of  the  PRISMS  virtual  reality 
flight  simulator,  including  the  research  requirements  for  such  a  device,  the  physical 
configuration  of  pilot  and  experimenter  stations,  control  devices,  HMD,  head- 
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tracker,  flight  model,  digital  terrain  data,  sound  and  video  systems,  data  recording 
capabilities,  and  user  interface. 

Section  6  discusses  the  formal  experiment  carried  out  with  14  Apache  pilots  at 
the  211th  Aviation  Regiment  at  West  Jordan,  Utah,  exploring  new  earth-fixed 
symbols  for  indicating  tactical  positions  critical  to  Army  rotorcraft  missions.  The 
experiment  was  most  effective  in  demonstrating  the  overwhelming  advantages  of 
the  new  earth-fixed  symbol  types.  The  accuracy  of  position-finding  in  the  terrain 
and  engagement  area  fire  sector  identification  were  enormously  improved  through 
the  display  of  virtual  waypoint  and  engagement  area  symbols. 

Section  7  presents  the  results  of  the  knowledge  acquisition  sessions  conducted 
with  the  Apache  pilots  immediately  following  the  experiments.  Several  new  HMD 
symbols  were  demonstrated  and  evaluated  by  the  pilots,  including  new  symbols  for 
presenting  slope  landing  data,  wind  speed  and  direction,  required  speed  for  accurate 
arrival  time,  threat  weapon  direction,  and  flight  path  prediction.  All  of  these  new 
symbols  were  judged  to  be  quite  valuable,  especially  if  intelligently  moded.  The 
pilots  suggestions  for  intelligent  moding  opportunities  are  also  described.  The 
section  concludes  with  the  results  of  a  survey  rating  the  importance  of  information 
elements  potentially  displayable  as  HMD  symbols,  as  a  guide  for  prioritizing  future 
research. 

In  addition,  the  project  has  also  demonstrated  the  relative  ease  with  which  the 
PRISMS  simulator  can  be  used  to  construct  and  edit  experimental  sessions,  add  and 
improve  symbology  features  and  behaviors,  provide  realistic  terrain  and  objects,  and 
create  an  extensive  range  of  performance  measures — all  in  a  package  that  is  easily 
transportable  to  the  field.  PRISMS  has  not  only  fulfilled  its  project  objectives,  but 
will  continue  to  provide  a  powerful  but  inexpensive  simulator  for  research  and 
training  for  many  years  to  come. 
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Section  1:  Introduction  and  Project  Summary 


This  Final  Technical  Report  is  submitted  in  accordance  with  the  provisions  of 
Contract  No.  D A AJ 02-96-C-0042,  issued  by  the  Aviation  Applied  Technology  Direc¬ 
torate,  U.S.  Army  Aviation  and  Troop  Command,  Fort  Eustis,  VA.  The  report  describes 
a  project  supported  by  a  Small  Business  Innovation  Research  (SBIR)  Program  Phase  II 
contract.  The  project  has  been  conducted  to  explore  the  development  of  an  intelligent 
symbology  management  system  for  helmet-mounted  displays  in  rotorcraft  (SBIR  Topic 
OSD  95-002,  Phase  II). 

Background 

Military  helicopter  pilots  experience  high  workload  demands  as  a  normal 
condition  of  combined  aviation,  navigation,  communication  and  mission  management 
requirements.  Missions  performed  by  nap-of-the-earth  (NOE)  flight  in  threat  areas  at 
night  or  in  adverse  weather  conditions  are  extremely  demanding  upon  all  of  the  pilot's 
perceptual,  cognitive,  and  psychomotor  resources.  Although  there  is  a  broad  awareness 
of  these  problems,  the  continuing  addition  of  new  avionics  capabilities  is  potentially 
leading  to  ever-greater,  rather  than  reduced,  workloads. 

In  particular,  the  development  of  avionics  for  presentation  of  symbology  on  a 
helmet-mounted  display  (HMD)  will  greatly  increase  the  pilot's  information 
management  requirements.  New  symbols  for  flight  control,  sensor  systems,  weapons 
management,  target  acquisition,  navigation,  data  linking,  aircraft  survival  equipment, 
and  battlefield  awareness  aids  are  all  competing  for  space  on  the  HMD.  The  HMD, 
originally  developed  to  reduce  workload  and  permit  pilots  to  keep  their  heads  up  and 
eyes  out  of  the  cockpit,  could  potentially  be  cluttered  with  symbology  that  obscures  the 
real-world  terrain.  As  a  result,  the  management  of  HMD  symbology  could  rapidly 
becoming  a  high-workload  task  in  itself. 

AH-64  Apache  HMD 

Currently,  the  only  U.S.  production  aircraft  that  makes  use  of  an  HMD  and  a  flight 
symbology  graphics  generator  is  the  AH-64  Apache  helicopter.  The  Apache  Pilot  Night 
Vision  System  (PNVS)  provides  a  head-steered  Infrared  (IR)  image  overlaid  with  flight 
symbology  to  the  pilot  using  a  monocular  HMD  designated  as  the  Integrated  Helmet 
and  Display  Sighting  System  (IFIADSS),  as  shown  in  Figure  1-1.  Although  the  IR  image 
is  head-steered,  most  of  the  flight  symbology  is  independent  of  head  position,  and 
appears  as  if  "painted"  on  the  HMD  screen.  Thus,  the  IHADSS  is  comparable  to  a 
miniature  Head-Up-Display  mounted  on  the  helmet. 
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Figure  1-1.  IHADSS  helmet  mounted  display  system. 

Pilots  have  often  noted  that  the  symbology  moding  approach  used  in  the  AH-64  is 
less  than  ideal,  both  in  terms  of  its  specific  organization  of  symbols,  and  in  the  manual 
mode  switching  actions  required  of  the  pilot.  The  IHADSS  uses  four  moding  options: 
Hover,  Bob-up,  Transition,  and  Cruise.  The  flight  symbology  mode  switch,  located  on 
the  cyclic  handgrip,  is  a  three-position  momentary  switch  which  permits  the  selection  of 
the  four  different  flight  symbology  modes. 

Emphasis  on  the  Pilot 

The  AH-64  is  a  tandem  aircraft  in  which  the  pilot  sits  in  back  and  the  co-pilot 
gunner  (CPG)  occupies  the  front  seat.  Both  pilots  wear  the  IHADSS  and  have  access  to 
a  number  of  common  symbol  elements.  However,  the  pilot  is  the  only  one  who  views 
PNVS  symbology  as  part  of  his  aircraft  controlling  duties.  The  CPG  is  responsible  for 
viewing  terrain,  navigating,  and  operating  the  aircraft's  complex  weapon  systems 
(guns,  rockets,  missiles).  In  support  of  these  duties,  the  CPG  uses  a  different  sensor 
system,  the  Target  Acquisition  Display  System  (TADS),  and  spends  a  good  portion  of 
each  mission  head  down,  with  the  IHADSS  monocle  pulled  back.  Consequently,  his 
present  use  of  IHADSS  symbology  is  limited,  and  his  requirements  are  quite  different 
from  those  of  the  pilot.  In  order  to  focus  our  planned  development  of  an  intelligent 
management  system  on  the  heaviest  user  of  HMD  symbology,  we  confined  our 
investigations  to  establishing  the  pilot's  requirements  for  improved  moding  and 
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symbology  presentation  methods  on  the  IHADSS.  However,  many  of  the 
recommendations  and  conclusions  of  our  study  will  apply  equally  to  the  CPG. 

Anacapa  Sciences'  Philosophy  of  Symbology  Research 

The  Difficulties  of  Display  Design 

There  is  broad  agreement  in  the  avionics  design  community  that  HMD  symbology 
development  issues  are  very  complex  because  of  the  many  different  kinds  of 
requirements  involved.  Operational  requirements  determine  the  information  that  must 
be  presented  to  the  pilot;  technological  advances  impact  the  potential  range  of  presen¬ 
tation  methods;  human  perceptual  and  cognitive  capabilities  and  limitations  constrain 
symbol  characteristics;  and  the  history  of  avionics  development  has  imposed  a  number 
of  standards,  methods,  and  expectations  on  the  design  process  itself. 

Given  the  multidisciplinary  character  of  the  design  requirements,  it  is  logical  to 
assume  that  similarly  constructed  teams  might  offer  the  best  approach  for  addressing 
the  problems  of  symbology  development.  Unfortunately,  there  is  no  set  of  guidelines 
readily  available  to  assist  the  multidisciplinary  design  team  in  the  selection  of  the 
optimal  information  presentation  methods,  nor  is  there  even  a  commonly-accepted 
model  of  symbology  development.  For  example,  Newman  &  Haworth  (1994)  in 
discussing  shortcomings  of  HMD  symbology  state  that  "The  underlying  causes  are  (1) 
the  absence  of  a  logical,  organized  design  methodology  and  (2)  the  absence  of  a  test  and 
evaluation  criteria."  In  addition,  Weintraub  &  Ensing  (1992),  in  their  review  of  head-up 
display  issues,  note  regretfully  that  "Improved  symbology  solutions  cannot  now  be 
designed  and  constructed  from  theory,  or  from  a  set  of  design  principles.  Present 
symbology  design  seems  to  be  partly  science,  partly  technology,  and  partly  cut  and 
try." 

As  a  result,  the  design  process  often  seems  more  of  an  institutionalized  struggle 
among  groups  having  very  different  objectives  than  the  goal-oriented  interactions  of  a 
design  "team."  Salter  (1991),  for  example,  describes  the  fundamental  nature  of  the 
interface  design  task  as  "better  conceived  of  as  the  reconciliation  of  conflicting 
constraints  than  as  the  application  of  clear-cut  rules."  Not  surprisingly,  the  absence  of  a 
clear  model  for  the  symbology  design  process  is  closely  related  to  the  lack  of  agreement 
on  the  definition  of  a  broadly  accepted  HMD  symbology  set.  The  true  benefit  of  a 
clearly  articulated,  top-down  symbology  design  process  lies  in  the  subsequent  phases  of 
display  design,  in  which  complexity  can  be  systematically  managed;  the  products  of 
thinking  can  be  readily  understood,  evaluated,  and  modified;  and  gaps  in  knowledge 
made  evident.  It  is  in  the  light  of  this  philosophy  that  our  model  of  the  symbology 
development  process  was  constructed. 

The  Symbology  Development  Model 

Over  the  past  several  years  of  research  in  the  area  of  symbology  development,  we 
have  developed  a  conceptual  model  of  critical  issues  that  must  be  addressed,  and  their 
hierarchical  relationships  (e.g.,  Rogers  &  Spiker,  1995).  This  model  is  shown  in 
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simplified  form  in  Figure  1-2,  below.  Although  the  actual  process  is  vastly  more 
complex  than  that  shown  in  this  diagram,  depicting  the  relationships  among  the  major 
issues  nevertheless  provides  a  very  useful  structure  for  organizing  the  available 
knowledge  on  symbology  development. 


Figure  1-2.  The  symbology  development  model,  in  simplified  form. 

The  diagram  shows  a  logical  sequence  of  issues  to  be  addressed,  yet  it  is  not 
anticipated  that  these  events  will  actually  follow  in  a  smooth  chronological  order; 
continuous  iteration  is  to  be  expected,  and  desired.  Nevertheless,  conceptualizing  the 
process  in  this  manner  is  quite  helpful  in  recognizing  the  utility  and  role  of  new 
information  as  it  becomes  available,  and  in  anticipating  the  potential  benefits  of  this 
information  for  "downstream"  activities.  The  focus  of  the  Phase  II  research  was  on  the 
box  entitled  "Develop  Rules  for  Display  Location  and  Moding,"  serving  to  underscore 
that  the  research  must  thus  address  all  of  the  "upstream"  activities.  Our  methods  for 
addressing  each  of  these  activities  is  described  below  and  in  the  following  section  of  the 
report. 

Figure  1-2  also  shows  that  the  development  of  symbology  systems  is  responsive  to 
two  major  influences:  the  information  to  be  conveyed,  shown  at  the  left  of  the  figure, 
and  the  current  knowledge  regarding  human  performance  with  display  techniques, 
shown  at  the  right  of  the  figure.  The  information  requirements  must  be  determined  by 
analytical  methods,  and  are  influenced  by  the  mission  of  the  aircraft,  special  operational 
problems  experienced  by  the  pilot,  and  the  particular  set  of  equipment  to  be  used  in 
performing  the  mission  and  meeting  the  special  problems.  These  analyses  identify  the 
specific  information  elements  that  must  be  presented  to  the  pilot,  and  the  attributes  of 
this  information,  such  as  "spatial"  or  "numeric." 

The  human  performance  knowledge  includes  principles  from  studies  of  human 
perception  and  cognition,  aviation  design  standards,  "lessons  learned"  from  flight  tests, 
and  guidelines  from  human  factors  engineering  and  related  disciplines.  The  nature  of 
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the  human  performance  knowledge  influences  the  information  requirements  analyses 
as  well  as  the  development  of  the  display  rules  by  identifying  elements  and  attributes  of 
potential  design  importance  based  on  human  information  processing  capabilities  and 
limitations. 

Need  for  Multiple  Approaches 

For  many  years,  Anacapa  has  performed  a  variety  of  research  studies  of  avionics 
symbology  development  for  fixed  wing  and  rotary  wing  aircraft.  Based  on  our 
experiences,  we  have  come  to  believe  that  there  is  no  single  "true  path"  to  display  and 
symbology  improvement,  and  that  any  single  approach  is  almost  certainly  doomed  to 
producing  minor  symbology  improvements,  if  not  outright  failure.  It  is  imperative  that 
parallel  methodological  paths  be  followed  to  include  the  knowledge  that  can  be  gleaned 
from  different  fields  of  expertise  and  forms  of  evaluation.  As  shown  in  Figure  1-3  the 
knowledge  from  three  communities  must  be  acquired  and  integrated  in  the  design 
process.  In  addition,  a  variety  of  tools  must  be  brought  to  bear  that  effectively 
integrates  the  information  from  these  communities. 


SCIENTIFIC  COMMUNITY  \ 

•  Human  sensory  capabilities  ! 

•  Human  cognitive  limitations  j 

•  Environmental  effects 


ENGINEERING  COMMUNITY  ! 

•  New  technological  capabilities j 

•  Key  design  features  and  costs  | 

•  Integration  feasibility 


I  PILOT  COMMUNITY 
I  •  Mission  requirements 
I  •  Cockpit  functions  and  tasks 


L 


■  Workload  reduction  goals 


EXAMPLES  OF  TOOLS  FOR  SYMBOLOGY  DEVELOPMENT 
’  Controlled  experimental  methods  •  Focused  observation  sessions  •  Static  and  dynamic  simulation 

>  Functional  requirements  analysis  •  Expert  knowledge  elicitation  •  Cockpit  workload  analysis 

>  Structured  interview  techniques  •  Quantitive  survey  construction  •  Human  engineering  analyses 


Figure  1-3.  Examples  of  the  range  of  expertise  and  methods  required  for  good 
symbology  design. 

Symbology  development  requires  information  from  the  scientific,  engineering,  and 
pilot  communities.  Each  of  the  communities  offers  important  knowledge,  yet  each  has 
its  limitations  if  used  without  supporting  data  from  other  sources  of  data  and  insight. 
For  example,  new  technological  capabilities,  while  impressive,  may  offer  nothing  to 
improve  a  pilot's  performance,  and  may  even  increase  his  workload.  Vision  or 
cognition  research  may  be  inapplicable  if  it  does  not  fully  consider  limitations  to  display 
system  capabilities  or  the  actual  nature  of  the  operational  tasks  performed.  The  pilot's 
absorption  with  expertise  in  certain  tasks  may  become  irrelevant  with  new,  intelligent, 
avionics  systems. 

In  like  fashion,  the  tools  themselves  have  advantages  and  disadvantages. 
Controlled  experiments  are  powerful  stimulators  of  information  and  insight,  but  the 
utility  of  their  results  depends  upon  having  selected  variables  that  reflect  an 
understanding  of  the  pilot's  functions  and  tasks.  Expert  knowledge  elicitation,  in 
isolation,  cannot  serve  to  develop  new,  better  techniques  for  display  of  integrated 
information.  Task-analytic  methods  are  necessary  for  organization  of  knowledge,  but 
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do  not  lead  to  creative  solutions.  In  short,  we  believe  that  symbology  development  and 
intelligent  symbology  management  research  requires  the  flexible  and  creative 
application  of  a  broad  array  of  techniques,  and  that  a  clear,  direct  path  to  innovative 
solutions  cannot  be  completely  planned  in  advance. 

Need  for  a  Sophisticated ,  Low-Cost  HMD  Research  Simulator 

It  became  evident  that  one  necessary  step  along  the  path  was  the  development  of 
an  HMD  research  simulator  that  could  be  used  to  aid  in  integrating  the  knowledge  from 
the  three  communities,  demonstrating  established  problems  and  their  candidate 
solutions,  and  applying  the  range  of  research  tools  in  the  most  meaningful  ways.  On 
one  hand,  we  had  to  acknowledge  that  a  number  of  research  helicopter  simulators 
already  existed  and,  like  the  Crew  Station  R&D  Facility  (CSRDF)  at  NASA  Ames 
Research  Center  and  the  Simulator  Training  Research  Advanced  Testbed  for  Aviation 
(STRATA)  at  the  Army  Aviation  Center,  have  served  as  testbeds  for  valuable  studies  in 
recent  years. 

The  use  of  such  powerful  simulators,  however,  has  been  accompanied  by  certain 
disadvantages.  In  general,  they  are  extremely  expensive  to  construct,  require 
specialized  technical  support  personnel,  are  time-consuming  to  reprogram  for  new 
applications,  and  are  certainly  not  portable  to  other  locations.  These  simulators  are  also 
in  great  demand  for  major  research  projects,  such  as  the  LHX  studies  of  past  years  at  the 
CSRDF  and  the  Air  Warrior  research  now  underway  at  the  STRATA  facility.  Despite 
the  surge  of  interest  in  HMD  display  systems,  it  is  difficult  for  small  research  groups  to 
obtain  access  to  these  costly  and  tightly  scheduled  simulators. 

We  believed  that  the  solution  to  these  problems  was  in  the  development  of  a 
sophisticated,  but  relatively  low-cost  HMD  research  simulator.  Not  only  would  such  a 
device  permit  HMD  simulator  access  to  more  research  agencies,  but  if  properly 
designed,  would  provide  a  number  of  additional  advantages.  We  planned  to  develop  a 
low-cost  HMD  research  simulator  to  provide  the  capabilities  necessary  to  show  HMD 
symbology  in  screen-fixed,  aircraft-fixed,  and  earth-fixed  frames  of  reference,  along 
with  a  gaming  area  of  realistic  terrain  and  facilities  for  demonstration,  knowledge 
acquisition,  experimental  control,  and  data  recording. 

It  seemed  clear  that  an  immersive  approach  with  an  opaque  visor  could  be  used, 
saving  costs,  yet  providing  an  effective  virtual  reality  experience.  It  could  be  paired 
with  an  accurate  head  tracker  so  that  symbology  would  be  positioned  and  made  to 
behave  appropriately  based  on  the  user's  head  movements.  Such  a  system  could  be 
configured  to  simulate  flight  through  terrain  generated  from  digital  terrain  elevation 
data  (DTED).  It  could  also  employ  some  representative  versions  of  helicopter  cyclic  and 
collective  controls,  and  a  capability  of  realistic  flight  through  the  terrain  for 
demonstrations,  experiments  and  training  purposes. 

Furthermore,  we  believed  that  it  could  be  designed  for  flexible  use  so  that  indi¬ 
vidual  scientists  would  find  it  easy  to  incorporate  and  demonstrate  symbology  ideas, 
easy  to  program  for  experiments,  and  easy  to  transport  to  meetings,  military  facilities, 
and  other  sites  for  clear  communication  of  symbology  behavior  and  knowledge 
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acquisition  sessions.  In  short,  we  were  certain  that  a  low-cost  HMD  research  simulator 
could  be  constructed  to  serve  us  well  in  conducting  the  Phase  II  effort  as  well  as  in 
providing  an  innovative  product  that  could  be  used  by  scores  of  research  and  training 
organizations  unable  to  afford  multi-million  dollar  simulators. 

Objectives  of  the  Phase  II  Research  Effort 

The  overall  objective  of  the  Phase  II  effort,  was  to  develop  and  evaluate  an 
innovative  and  intelligent  information  presentation  system  for  an  HMD  in  a  synthetic 
environment.  The  subordinate  objectives,  based  upon  our  philosophy  of  symbology 
system  design,  included  the  following: 

(a)  Define  the  specific  information  elements  that  should  be  presented  via  the 
HMD  to  support  pilot  tasks,  especially  as  aircraft  flight  regimes  and  tactical 
requirements  change  during  the  course  of  a  mission. 

(b)  Develop  a  powerful,  portable,  intelligent  symbology  management  HMD 
simulator  to  permit  rapid  and  effective  identification  and  evaluation  of 
candidate  intelligent  prioritization  and  filtering  of  flight  mode  information. 

(c)  Develop  and  test  innovative  flight  and  mission  information  symbology 
management  concepts  on  the  portable  simulator,  realistically  demonstrating 
the  most  promising  concepts  for  providing  timely  information  to  the  pilot. 

Underlying  goal  of  broad  and  continuing  applicability.  In  addition  to  the  specific 
objectives  stated  above,  we  believed  that  an  underlying  goal  of  the  project  was  that  the 
"innovative  intelligent  symbology  management  system"  must  be  applicable  beyond 
any  one  specific  aircraft.  Thus,  although  we  had  chosen  to  focus  on  the  characteristics 
of  the  AH-64  aircraft  because  of  the  availability  of  SMEs,  the  products  of  this  Phase  II 
effort  were  most  definitely  to  extend  beyond  the  AH-64  aircraft  and  equipment.  The 
extended  applicability  of  this  work  was  to  take  two  different  forms:  first,  the  broad 
relevance  of  research  results  and  second,  the  continuing  value  of  the  research 
methodology: 

(1)  Broad  relevance  of  research  results.  The  results  themselves  were  to  offer  not  only 
specific  HMD  symbology  solutions  for  the  AH-64,  but  include  broadly  generalizable 
concepts  that  would  be  applicable  to  a  range  of  current  and  future  aircraft.  For 
example,  although  a  new  type  of  warning  symbol  might  be  designed  to  appear  in 
somewhat  different  colors,  shapes,  or  locations  in  various  aircraft,  the  recognition  that 
the  information  element  is  critical  and  that  certain  events  can  be  sensed  to  permit  its 
automatic  presentation  should  be  useful  in  multiple  aircraft  types. 

(2)  Continuing  value  of  the  research  methodology.  Because  the  information  content  of 
the  HMD  will  vary  enormously  given  the  different  aircraft  mission  equipment 
packages,  or  MEPs  (i.e.,  sensors,  weapons,  target  acquisition  methods,  navigation 
systems,  communication  systems,  aircraft  survival  equipment,  and  battlefield 
awareness  aids),  no  single  set  of  intelligent  symbology  management  techniques  can  be 
expected  to  fully  meet  die  requirements  of  new  systems.  For  this  reason,  a  portion  of 
our  Phase  II  effort  was  to  be  devoted  to  devising  an  efficient  methodology  for  iden- 
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tifying  and  evaluating  intelligent  symbology  management  opportunities  as  they  arise 
from  the  addition  of  new  aircraft  to  the  field,  or  new  equipment  items  to  existing 
aircraft. 

Overview  of  the  Phase  II  Technical  Approach 

The  overall  approach  to  the  Phase  II  effort  is  shown  in  Figure  1-4,  below.  In  Task 
1,  based  on  our  successes  in  the  Phase  I  effort,  we  continued  and  greatly  broadened  our 
efforts  in  identifying  innovative  symbology  management  concepts.  In  Task  2  we 
expanded  our  analyses  of  mission  information  requirements  to  include  a  much  larger 
array  of  pilot  functions  and  tasks  than  had  been  addressed  during  Phase  I.  The  products 
of  Task  1  and  2  supported  Task  3,  a  concurrent,  parallel  effort  to  develop  a  relatively 
low-cost,  but  sophisticated  intelligent  symbology  simulator,  incorporating  an 
immersive  HMD  and  head-tracker  system  with  multiple  frames  of  reference  for 
symbology  presentation. 


Figure  1-4.  A  diagram  of  the  five  tasks  in  the  Phase  II  project. 

Our  new  Pilot-Rotorcraft  Intelligent  Symbology  Management  Simulator  (PRISMS) 
was  designed  to  present  rapidly  reconfigurable  screen-fixed,  aircraft-fixed,  and  earth- 
fixed  symbology  in  a  variety  of  mission  scenarios.  Thus,  the  complexities  of  symbol 
motion  dynamics  are  realistically  presented  to  the  SMEs  and  their  ideas  can  be 
incorporated  for  "quick-look"  evaluations.  The  simulator  software  was  designed  to  be 
capable  of  direct  use  in  helicopter  flight  tests  as  well.  Tasks  1, 2,  and  3  supported  Task 
4,  the  application  of  the  HMD-based  simulator  to  knowledge  acquisition  sessions  with 
SME  pilots.  Because  the  simulator  is  portable,  we  were  able  to  transport  it  to  a  suitable 
Army  site  for  experimental  and  knowledge  acquisitions  sessions  with  Apache  pilots. 

In  addition  to  its  symbology  presentation  capabilities,  the  PRISMS  simulator  was 
specifically  designed  to  manage  controlled  experiments,  recording  such  data  as  RMS 
error  in  altitude,  airspeed,  and  flight  path,  as  well  as  response  times  for  target  or  event 
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detection.  These  capabilities,  and  many  more,  were  put  to  use  in  the  ground-based 
intelligent  symbology  management  evaluations  of  Task  5.  Experiments  conducted 
during  Task  5  were  used  to  identify  the  most  promising  configuration  of  intelligent 
symbology  management  characteristics  and  to  explore  the  effectiveness  of  various 
moding  rules  employed  to  control  the  presentation  and  behavior  of  the  new  symbols. 

Loss  of  Flight  Testing  Activities  in  the  Phase  II  Project 

The  original  description  of  the  Army  SBIR  Topic  OSD95-002,  called  for  preliminary 
evaluations  of  intelligent  information  presentation  concepts  for  an  HMD  to  be 
performed  in  “both  ground  and  in-flight  simulation  to  verify  improvement  potential. 
Complete  definition  of  intelligent  symbology  management  characteristics  of  the  most 
promising  configuration  will  be  verified  in  flight  tests  on  helicopters  with  HMD  systems” 
(italics  added). 

We  were  pleased  to  see  in-flight  testing  called  out  as  a  research  requirement;  in  our 
view,  there  is  no  symbology  evaluation  methodology  that  is  more  convincing  than  the 
measurement  of  aviators’  performance  in  flight,  provided  that  the  previous  steps  in  the 
symbology  design  process  have  been  effectively  carried  out.  We  were  particularly 
eager  to  validate  our  intelligently-moded  symbology  using  the  FLITE  (Flying 
Laboratory  for  Integrated  Test  and  Evaluation)  aircraft  as  Government-Furnished 
Equipment  (GFE). 


Figure  1-5.  The  Flying  Laboratory  for  Integrated  Test  and  Evaluation. 

Unfortunately,  due  to  Army  operational  requirements,  FLITE  became  unavailable 
during  the  project’s  period  of  performance.  Instead,  all  tests  were  to  be  performed 
using  our  PRISMS  simulator.  Although  disappointing,  this  change  had  less  practical 
effects  than  it  might  seem.  All  of  the  symbology  testing  we  had  envisioned  for  FLITE 
could  easily  be  performed  on  the  PRISMS  simulator. 

Organization  of  the  Report 

The  remainder  of  this  report  is  subdivided  into  six  major  sections.  Section  2 
describes  the  activities  undertaken  during  the  Phase  I  SBIR  effort,  including  the  initial 
information  requirements  analyses,  preliminary  interviews  with  Apache  subject-matter 
experts,  flights  in  the  Army’s  STRATA  simulator,  the  development  of  an  animated 
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laptop  symbology  simulator,  and  the  results  of  interviews  and  a  survey  of  instructor 
pilots  at  the  Army  Aviation  Center  at  Fort  Rucker,  Alabama. 

Section  3  describes  the  conduct  and  results  of  Task  1,  examining  the  symbology 
management  issues  of  mode  switching  and  information  organization.  The  current 
pilot-operated  mode  switching  methods  are  contrasted  with  alternate  psychomotor 
controls  such  as  head-  and  eye-aimed  controls,  voice-actuated  controls,  touch-sensitive 
panels,  and  virtual  hand  controllers.  This  section  also  compares  the  symbology  moding 
strategies  of  the  AH-64  Apache,  RAH-66  Comanche,  and  the  ANVIS/HUD  night  vision 
system  and  describes  the  potential  reductions  in  pilot  workload  with  the  development 
of  an  intelligent  system  to  control  the  presentations  of  individual  symbols. 

Section  4  identifies  the  nature  of  Army  mission  management  and  communication 
tasks,  showing  the  tremendous  information-handling  burden  carried  by  the  Army 
aviator  and  describes  HMD  solutions  to  the  current  shortcomings  in  mission  manage¬ 
ment,  focusing  on  the  concept  of  "augmented  reality"  as  the  key  to  workload  reduction 
and  situation  awareness  enhancement.  This  section  also  describes  Task  2,  the  mission 
information  requirements  analysis,  the  methods  of  identifying  potential  HMD  symbol- 
ogy  applications,  and  a  list  of  124  information  elements  identified  as  potentially  useful 
on  an  HMD  display. 

Section  5  presents  the  design  and  development  of  the  PRISMS  simulator  (Task  3) 
including  the  requirements  for  such  a  device,  the  desired  capabilities,  the  physical 
configuration  of  pilot  and  experimenter  stations,  control  devices,  HMD,  head-tracker, 
flight  model,  digital  terrain  data,  sound  and  video  systems,  data  recording  capabilities, 
and  user  interface. 

Section  6  discusses  the  formal  experiment  carried  out  with  14  Apache  pilots  at  the 
211th  Aviation  Regiment  at  West  Jordan,  Utah,  exploring  new  earth-fixed  symbols  for 
waypoint  markers,  engagement  areas,  and  other  tactical  positions  critical  to  Army 
rotorcraft  missions.  The  methods,  procedures,  experimental  design,  and  results  of  the 
experiment  are  described  in  detail. 

Section  7  describes  the  results  of  knowledge  acquisition  sessions  conducted  with 
the  Apache  pilots  after  demonstrations  of  several  new  HMD  symbols  were  conducted 
immediately  following  the  experiment.  The  section  concludes  with  the  results  of  a 
survey  of  the  importance  of  information  elements  potentially  displayable  as  HMD 
symbols. 
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Section  2:  Review  of  Phase  I  Activities 


The  following  pages  describe  our  Phase  I  project  activities,  techniques,  and 
outcomes.  Although  this  section  is  not  essential  to  the  description  of  the  Phase  II 
project,  this  background  information  is  offered  here  as  a  foundation  for  the  tasks 
undertaken  in  Phase  II. 

The  overall  objective  of  the  Phase  I  effort  was  to  determine  the  feasibility  of  an 
innovative  intelligent  symbology  management  system  for  an  Army  Rotorcraft 
HMD.  Subordinate  objectives  included: 

(a)  Determine  elements  of  information  necessary  to  support  pilot  tasks  as 
aircraft  flight  regimes  change  during  the  course  of  a  mission. 

(b)  Identify  and  evaluate  innovative  flight  and  mission  information 
symbology  management  concepts,  highlighting  the  most  promising 
concepts  for  providing  timely  information  to  the  pilot. 

(c)  Develop  a  portable,  intelligent  symbology  management  simulator  to 
permit  the  comparison  of  candidate  intelligent  information  prioritiza¬ 
tion/  filtering  techniques. 

(d)  Demonstrate  potential  increase  in  pilotage  and  mission  effectiveness  and 
the  feasibility  of  the  intelligent  symbology  management  system  concepts. 

Having  successfully  concluded  the  Phase  I  effort,  and  with  the  luxury  of  hind¬ 
sight,  we  believe  these  objectives  to  have  been  well-chosen.  Not  only  were  we  able 
to  attain  these  goals,  but  they  had  jointly  led  to  a  broader  and  deeper  understanding 
of  specific  attack  helicopter  mission  requirements  and  greater  degree  of  insight 
regarding  how  these  requirements  can  be  supported  through  advanced  HMD 
systems  with  intelligent  symbology  management. 

During  the  initial  months  of  our  Phase  I  effort  we  primarily  addressed  the  first 
two  objectives.  We  identified  the  specific  elements  of  information  needed  to 
support  aviation  and  navigation  tasks  and  determined  how  requirements  for  these 
information  elements  changes  during  the  course  of  the  mission.  Because  of  the 
complex  nature  of  the  shifting  requirements,  we  found  it  necessary  to  extend  our 
analyses  beyond  flight  control  tasks  in  isolation,  and  begin  to  examine  the  tactical 
situations  in  which  these  tasks  are  performed.  In  addition,  we  identified  several 
innovative  flight  and  mission  information  symbology  management  concepts,  and 
explored  some  promising  concepts  for  displaying  information  to  the  pilot  only 
when  it  is  needed. 

During  the  subsequent  months  of  the  Phase  I  effort,  and  armed  with  the 
information  requirements  and  symbology  management  concepts,  we  turned  to 
objectives  (c)  and  (d).  We  focused  on  the  development  of  a  laptop  HMD  intelligent 
symbology  simulator  and  definition  of  evaluation  techniques  for  comparing  base¬ 
line  PNVS  moding  and  intelligent  symbology  management  approaches.  Extensive 


2-1 


preparations  were  made  for  interviews  with  very  experienced  Instructor  Pilots  (IPs) 
at  the  Aviation  Training  Brigade  at  the  Army  Aviation  Center  at  Fort  Rucker, 
Alabama. 

Research  Conducted  During  Phase  I 

Information  Requirements  Analyses 

We  began  the  project  by  performing  an  information  requirements  analysis  on  a 
limited  subset  of  mission  functions  to  determine  the  specific  elements  of  informa¬ 
tion  necessary  to  support  pilot  tasks.  The  purposes  of  the  analyses  were  to  extract  in¬ 
formation  directly  pertaining  to  potential  HMD  intelligent  symbology  management 
characteristics,  and  were  thus  focused  on  issues  perceived  to  be  most  likely  to  yield  a 
payoff.  The  analyses  were  undertaken  from  two  perspectives:  AH-64-specific  tasks 
and  generic  tasks.  The  advantage  of  studying  AH-64-specific  tasks  was  that  workload 
metrics  available  to  us  could  be  used  to  identify  demanding  visual  and  cognitive 
tasks  for  discussing  potential  solutions  with  subject  matter  experts.  The  advantage 
of  the  generic-task  approach  was  that  it  permitted  the  identification  of  potential 
symbology  elements  at  an  information  level,  rather  than  with  reference  to  some 
existing  piece  of  hardware  in  the  AH-64A. 

Examine  Apache  Workload  Prediction  Model  Data 

Anacapa  has  conducted  a  series  of  workload  prediction  studies  for  Army  air¬ 
craft,  including  the  Apache  (e.g.,  Hamilton,  1992;  Szabo  &  Bierbaum,  1986).  The 
hundreds  of  tasks  are  described  by  verb-object  pairs  (such  as  "Check  tailwheel 
switch")  and  by  workload  components  ratings  for  visual,  auditory,  cognitive,  and 
psychomotor  task  requirements  made  by  Apache  pilots.  Examination  of  the  detailed 
data  has  been  useful  in  identifying  tasks  having  high  visual  workload,  such  as:  "set 
accelerometer,"  "check  ACQ  SEL  switch,"  "check  aircraft  location,"  "check  laser 
code,"  "set  Doppler  mode  switch,"  "set  gun  burst  limits,"  "set  missile  mode  switch," 
"control  rate  of  descent,"  "check  standoff  range,"  "enter  target  coordinates,"  "set 
transponder  control  switch,"  "set  UHF  frequency  selector  switches,"  and  "set  Mode  4 
switch,"  among  many  others.  Many  of  the  tasks  rated  high  for  visual  or  cognitive 
workload  led  directly  to  fruitful  discussion  topics  during  the  expert  interviews. 

Select  Generic  Flight  and  Mission-Oriented  Segments  and  Functions 

We  then  selected  a  set  of  in-flight  activities  to  create  a  representative  range  of 
information  elements  and  pilot  information  processing  functions.  The  activities 
were  selected  to  include  events  leading  to  changing  symbology  needs  and  were 
sufficiently  demanding  so  that  potential  reductions  in  workload  could  ultimately  be 
verifiable  and  valuable.  In  order  to  ensure  that  supporting  subject  matter  experts 
would  be  available,  the  current  AH-64  tasks  served  as  the  pool  from  which  the 
analyzed  activities  were  drawn.  Nevertheless,  the  types  of  pilotage  and  mission 
functions  examined  were  representative  of  those  to  be  performed  by  aviators  in  the 
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foreseeable  future,  even  in  more  advanced  aircraft  and  using  more  powerful  avion¬ 
ics  capabilities,  so  that  our  findings  could  have  long-term  applicability. 

Because  Anacapa  had  previously  developed  detailed  task  and  workload 
analyses  in  support  of  the  AH-64  and  Longbow  aircraft,  the  basic  structure  of  the 
tasks  were  familiar  to  us,  saving  extensive  efforts  in  preparation  for  the  information 
requirements  analysis.  In  a  recent  top-down  analysis  (Hamilton,  Rogers,  &  Spiker, 
1995),  we  identified  more  than  200  functions  for  armed  attack  and  reconnaissance 
mission.  For  the  purposes  of  the  Phase  I  effort,  we  selected  a  representative  sample 
of  about  25%  of  the  relevant  functions  (based  on  HMD  symbology  impacts)  and 
examined  them  in  detail.  The  functions  included  all  of  the  most  applicable  flight 
activities  such  as  "perform  rolling  takeoff,"  "transition  to  hover,"  "transition  to 
contour,"  "transition  to  NOE,"  "fly  contour,"  "hover  masked,"  and  "fly  IMC."  In 
addition,  we  examined  a  sample  of  mission-oriented  tasks  such  as  "acquire  ground 
target  with  helmet,"  "review  threat  hazards,"  "fire  rockets,"  "fire  guided  missile," 
"assign  fire  zones  to  flight  members,"  and  "send  situation  report." 

Study  Results  of  Systematic  Information  Decomposition 

A  top-down  analytical  approach  was  used  to  establish  the  information 
requirements  for  the  selected  activities  through  systematic  hierarchical  decomposi¬ 
tion  of  the  representative  mission  segments  (e.g.,  takeoff,  NOE  flight)  into  functions 
(e.g.,  mask  aircraft,  perform  hover),  tasks  (e.g.,  control  airspeed,  select  weapons 
system),  requirements  (e.g..  Is  the  aircraft  in  trim?),  and  information  elements  (e.g., 
altitude,  fuel  quantity).  Because  of  the  limited  time  and  resources  of  a  Phase  I  effort, 
it  was  not  possible  to  study  every  component  of  a  mission,  so  we  used  interviews 
with  subject  matter  experts  (SMEs)  to  guide  us  to  the  high-payoff  activities. 

Descriptions  of  the  required  information  elements  were  reviewed  as  a  part  of 
these  analyses.  Each  element  was  described  in  terms  of  its  units  of  measurement, 
current  source,  and  other  attributes.  The  units  of  measurement  and  source  are 
important  in  determining  how  well  or  how  easily  this  information  can  be  sensed, 
not  only  by  the  pilot,  but  by  an  intelligent  symbology  management  system.  The 
attribute  recording  options  were  structured  to  help  identify  cues  indicating  that 
HMD  display  symbols  should  be  presented  on,  or  removed  from,  the  HMD  screen. 
For  example,  in  the  Comanche  design,  the  attribute  of  "airspeed"  is  used  to  control 
the  appearance  of  the  flight  path  vector  symbol. 

Initial  SME  Interviews 

The  top-down  analyses  were  supplemented  by  important  bottom-up  data  on 
symbol  requirements.  Given  that  an  AH-64  symbol  set  was  already  in  existence,  and 
familiar  to  hundreds  of  Army  aviators,  we  elected  to  take  advantage  of  this  pool  of 
SMEs  to  identify  situations  in  which  existing  symbols  are  clearly  required  or  not 
required  for  conduct  of  inflight  tasks.  These  interviews  focused  both  on  individual 
symbols,  as  well  as  on  the  four  display  modes  on  the  IHADSS:  Hover,  Transition, 
Cruise,  and  Bob-up.  In  the  initial  phase  of  interviewing  (during  October,  1995), 
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pilots  on  the  Anacapa  staff  were  queried,  in  order  to  develop  an  effective  portable 
symbology  simulator  and  design  appropriate  knowledge  acquisition  tools  for  use 
with  the  IPs. 

A  special  interview  booklet  was  prepared  to  summarize  the  available  docu¬ 
mentation  on  IHADSS  symbology  and  guide  the  SMEs  through  the  discussion  of 
each  symbol  element.  In  order  to  clarify  symbols  interactions,  the  IHADSS  symbols 
were  discussed  in  terms  of  seven  categories  of  information  as  shown  in  Table  2.1. 
The  complete  set  of  IHADSS  symbols  is  shown  in  Figure  2-1,  below. 


Table  2.1 

Organization  of  IHADSS  Symbols  by  Information  Category 


Information  Category 

IHADSS  Symbol 

1.  Position  and 
Movement 

Line  of  Sight  (LOS)  Reticle,  Airspeed  Digital  Readout, 
Velocity  Vector,  Acceleration  Cue,  Hover  Position  Box, 
Radar  Altitude  Vertical  Tape 

2.  Attitude /Altitude 

Radar  Altitude  Vertical  Scale,  Radar  Altitude  Digital 
Readout,  Horizon  Line,  Engine  Torque  Digital  Readout, 

Rate  of  Climb  Indicator,  Skid /Slip  Ball,  Lubber  Lines 

3.  Heading/Navigation 

Heading  Scale,  Fixed  Lubber  Line,  Command  Heading, 
Alternate  Sensor  Bearing 

4.  Central 

Cueing/Reference 

Head  Tracker,  Cueing  Dots,  Cued  LOS  Reticle 

5.  Peripheral 

Cueing  /  Ref  erence 

Field  of  Regard  Box,  Field  of  View  Box,  Cued  LOS  Dot 

6.  Weapons  Usage 

Rocket  Steering  Cursor,  Missile  Constraints  Box 

7.  High-Action  Display 

Sight  Status,  Weapon  Status,  Weapon  Control, 

Range /Range  Source 

Each  IHADSS  symbol  was  discussed  in  detail,  confirming  its  format,  typical 
location,  and  behavior.  The  specific  purpose(s)  of  each  symbol  were  confirmed,  and 
the  conditions  under  which  each  symbol  appeared  or  disappeared  were  noted.  Most 
of  the  symbols  are  always  present,  while  others  appear  only  in  one  of  the  four  pilot- 
controlled  modes  (Cruise,  Transition,  Hover,  or  Bob-up).  Still  others  appear  or 
disappear  according  to  special  rules.  For  example,  the  Radar  Altitude  Vertical  Tape 
disappears  above  200  feet  AGL,  and  reappears  at  180  feet  AGL.  The  characteristics  of 
each  symbol,  i.e.,  its  changes  in  content,  form,  or  movement  in  correspondence  with 
the  underlying  aircraft  performance,  was  discussed  in  depth  prior  to  scrutinizing  its 
behavior  in  simulated  flight.  The  findings  are  summarized  in  Appendix  1  (pages 
75-102)  of  the  Phase  I  report  (Rogers,  Spiker,  &  Asbury,  1996). 


2-4 


1.  LOS  Reticle 

2.  Alternate  Sensor  Bearing 

3.  Lubber  Line 

4.  Cueing  Dots 

5.  Command  Heading 

6.  Acceleration  Cue 

7.  Velocity  Vector 

8.  Heading  Scale 


9.  Cued  LOS  Reticle 

10.  Missile  Constraints 

11.  Radar  Altitude 

12.  Rate  of  Climb 

13.  Radar  Altitude  Scale 

14.  Radar  Altitude  Tape 

15.  Skid/Slip  Lubber  Lines 

16.  Skid/Slip  Ball 


17.  Cued  LOS  Dot 

18.  Field  of  View 

19.  Sensor  Field  of  Regard 

20.  High  Action  Display 

21.  Rocket  Steering  Cursor 

22.  Hover  Position  Box 

23.  Head  Tracker 

24.  Airspeed 


Figure  2-1.  The  IHADSS  symbols  shown  in  representative  positions. 


STRATA  Simulator  " Flights " 

Preparation  for  the  conduct  of  the  SME  interviews  included  use  of  the  sophisti¬ 
cated  STRATA  simulator  operated  by  the  U.S.  Army  Research  Institute  Aviation 
R&D  Activity  (ARIARDA),  at  the  U.S.  Army  Aviation  Center  at  Fort  Rucker, 
Alabama.  When  it  was  constructed  in  the  mid  1980's,  STRATA  incorporated  state- 
of-the-art  technologies  throughout  the  system,  providing  opportunities  for  innova¬ 
tive  research  in  many  areas.  STRATA,  designed  as  a  reconfigurable  research  testbed, 
also  provides  all  the  necessary  capabilities  for  an  outstanding  AH-64  simulator.  Its 
technologies  include  the  ESIG-1000  Image  Generator  (IG),  capable  of  producing 
highly  realistic  out-the- window  and  sensor  visuals;  the  Fiber-Optic  Helmet- 
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Mounted  Display  (FOHMD),  capable  of  presenting  high-resolution,  high-brightness 
images  from  the  IG;  and  the  Interactive  Tactical  Environment  Management  System 
(ITEMS),  capable  of  accurately  simulating  the  battlefield  environment. 


Figure  2-2.  A  view  of  the  STRATA  cockpit. 

Given  that  our  efforts  were  focused  on  the  IHADSS  and  PNVS  symbology 
issues,  the  STRATA  capabilities  were  specifically  configured  so  that  the  project  Prin¬ 
cipal  Investigator  could  experience  the  IHADSS  display  symbology  under  a  variety 
of  realistic  visibility  conditions,  and  during  a  range  of  flight  and  mission-oriented 
tasks. 

To  most  effectively  illustrate  the  behaviors  of  the  symbols,  the  Principal  Inves¬ 
tigator  was  given  an  opportunity  to  “fly”  the  simulated  AH-64  for  several  hours 
over  the  course  of  two  days,  performing  Hover,  Bob-up,  Transition,  and  Cruise 
mode  changes  and  other  flight  activities  in  rolling  Arizona  terrain,  while  closely 
observing  the  behavior  of  the  PNVS  symbology.  The  Apache  SME  in  the  other 
cockpit  of  the  simulator  clarified  critical  HMD  symbology  issues,  suggested  maneu¬ 
vers  for  demonstrating  symbol  dynamics,  and  set  up  the  systems  for  various  tactical 
engagements.  The  flights  included  conditions  of  daylight,  low  light,  low  light  over¬ 
laid  with  PNVS  imagery,  and  night  with  PNVS  imagery.  The  flights  were  alter¬ 
nated  with  interview  sessions  in  order  to  most  clearly  illustrate  the  issues  discussed 
during  the  interviews,  identify  situations  in  which  various  symbols  were  clearly 
required  or  not  required  during  flight,  and  specify  the  key,  potentially  sensed  events 
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(e.g.,  altitude,  distance,  time,  location,  or  equipment  use)  that  might  serve  as  corre¬ 
lates  of  these  changing  information  needs. 

The  Principal  Investigator  had  been  thoroughly  familiarized  with  the  IHADSS 
symbology  by  studying  the  AH-64  manuals  detailing  the  uses  and  behavior  of  the 
symbology,  and  from  SME  interviews  immediately  preceding  the  flights.  Neverthe¬ 
less,  the  insights  gained  from  a  few  hours  use  of  the  STRATA  simulator  clarified 
the  symbol  dynamics  and  interactions  to  a  much  greater  extent.  Furthermore,  the 
ability  to  effectively  discuss  the  symbology  was  vastly  extended  by  the  opportunity 
for  both  of  the  discussants  to  simultaneously  observe  and  comment  upon  the 
symbology  set  in  action. 

Development  of  PRISMS  I 

As  a  result  of  the  insights  gained  during  the  STRATA  sessions,  it  became 
apparent  that  SME  interviews  would  benefit  greatly  from  the  availability  of  a  PNVS 
symbology  simulator.  In  anticipation  of  a  need  for  such  a  simulator,  we  had  origi¬ 
nally  proposed  a  PRISMS  simulator  for  the  project  in  order  to  provide  a  series  of 
useful,  but  static,  graphics  for  presentation  during  the  interviews.  After  the 
STRATA  sessions,  we  performed  a  major  upgrade  of  the  PRISMS  software  to  add  a 
symbol  animation  capability  and  renamed  it  "PRISMS  I,"  in  expectation  of  addi¬ 
tional  versions.  PRISMS  I  was  first  programmed  in  the  VisualWorks  Smalltalk 
language  for  the  PC,  and  later  in  a  Macintosh-compatible  version.  The  animation 
features  assisted  enormously  in  clarifying  the  relationships  among  symbols  and 
their  movements. 

Because  a  primary  focus  of  Phase  I  had  been  the  determination  of  appropriate 
rules  to  support  a  rule-based  system  for  intelligent  symbology  management,  we 
stressed  the  development  of  PRISMS  I  features  for  knowledge  acquisition.  Thus, 
PRISMS  I  was  designed  not  only  as  an  intelligent  display-driving  device,  but  also  to 
include  specific  features  for  use  in  expert  interviews.  PRISMS  I  was  configured  with 
an  easy-to-use  dialogue  for  operation  by  Anacapa  or  Government  personnel.  The 
full  set  of  PNVS  symbols  was  provided  in  the  HMD  symbology  window,  and  the 
Bob-up,  Hover,  Transition,  and  Cruise  Modes  could  be  selected  by  clicking  a  button 
in  the  HMD  Control  Window,  as  shown  in  Figure  2-3.  The  system  could  also  be  set 
to  function  in  the  "Intelligent  Moding"  role,  responding  to  rules  set  in  the  PRISMS  I 
Rule  Editor. 

In  the  lower-right  window  (Fort  Rucker)  of  Figure  2-3  is  a  scrollable  set  of 
"Scenarios"  that  could  easily  be  constructed,  saved,  and  selected  by  the  operator.  The 
scenarios  were  used  to  input  attributes  (such  as  airspeed,  altitude,  bank  angle)  to  set 
up  the  symbology  window  for  simulating  various  activities,  such  as  NOE  flight  or 
target  acquisition.  These  same  attributes  could  be  used  as  "facts"  by  the  Intelligent 
Moding  system,  in  determining  the  appearance  and  decluttering  of  symbols. 
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PRISMS  -  Pilot/Rotorcraft  Intelligent  Symbology  Manag 


File  HMD  Symbols  Controls  Animation 


I  I 

A 


- -  T48>-^ 


PRISMS  HMD  Contri 


File  Edit 


-HMD  Symbol  Modes  — 
O  Houer  O  Bob-Up 

©TransitioiO  Cruise 
O  Intelligent  Moding 


i—  rFort  Rucker:- 


4  Contour  example 

5  Rocket  constraints 

6  Missile  constraints 
DEFAULT 

SPR1 


Figure  2-3.  The  PRISMS  I  Symbology  and  HMD  Control  Windows. 

Selecting  a  scenario  from  the  control  window  and  pulling  down  the  Edit  Menu 
in  the  HMD  Control  window  called  up  The  Scenario  Editor  Window,  which  listed 
all  of  the  attributes  and  current  values  of  the  attributes  for  the  selected  scenario,  as 
shown  in  the  left  of  Figure  2-4.  Each  attribute  could  be  set  in  the  Relation  Editor 
Window,  as  shown  at  the  right  of  Figure  2-4.  For  example,  the  "Wobble  Range"  and 
"Wobble  Increment"  entries  permitted  precise  control  over  the  range  and  speed  of 
animation  for  each  individual  symbol. 


Figure  2-4.  The  PRISMS  Scenario  Editor  and  Relation  Editor  Windows. 
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The  PRISMS  Rule  Editor  permitted  entry  of  sets  of  experimental  rules  for  the 
conditional  display  of  symbol  elements.  These  rules  were  used  to  control  the 
appearance  and  deletion  of  symbol  elements  in  the  Symbology  Window  when  the 
Intelligent  Moding  option  was  selected.  The  simplicity  of  the  user  interface,  shown 
in  Figure  2-5,  belied  the  power  of  the  intelligent  system  software  underlying  the 
Intelligent  Moding  capability.  For  a  simple  example.  Figure  2-5  shows  the  entry  of  a 
rule  reading  "IF  the  Aircraft  Roll  is  greater  than  5°,  THEN  the  Horizon  Line  is  ON." 
Any  number  or  rules  could  be  included  in  a  set,  and  any  number  of  sets  could  be 
stored.  For  purposes  of  exploratory  development,  these  rules  were  instantiated  by 
settings  in  the  scenarios. 


PRISMS  Rule  Set  Editor 


PRISMS  Rule  Editor 
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File  Edit 
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Figure  2-5.  The  PRISMS  Rule  Set  Editor  and  Rule  Editor. 


AH-64  Instructor-Pilot  Interviews 

Interviews  were  arranged  with  12  Instructor-Pilots  (IPs)  at  Fort  Rucker, 
Alabama,  during  February,  1996.  The  IPs  were  all  from  the  1/14  Aviation  Regiment, 
and  instructed  the  AH-64A  Qualification  Course.  All  of  the  IPs  had  extensive  expe¬ 
rience  in  operational  units  before  their  instructional  assignment,  averaging 
approximately  1900  rotary-wing  flight  hours  and  over  1100  hours  in  the  AH-64.  The 
IPs  were  interviewed  individually,  with  sessions  lasting  90  minutes  to  2  hours. 

Prior  to  the  interviews,  we  had  prepared  comprehensive  knowledge  acquisi¬ 
tion  booklets  with  dozens  of  discussion  topics  based  on  our  prior  studies  of  mission 
requirements  and  near-term  technology  improvements.  The  booklets  were 
prepared  through  extensive  reviews  of  Anacapa's  attack  helicopter  task  analysis 
data,  pertinent  published  literature,  and  our  own  in-house  archives  to  identify 
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mission  segments  and  tasks  that  might  be  better  performed  with  the  aid  of  new 
HMD  symbology  capabilities.  We  also  reviewed  our  prior  studies  of  new  mission 
equipment  packages  (MEPs)  that  will  permit  (or  require)  new  HMD  symbology. 
Because  of  the  breadth  of  the  investigations,  there  was  insufficient  time  for  all  of  the 
IPs  to  discuss  each  of  the  topics.  However,  each  topic  was  addressed  by  more  than 
one  IP,  and  some  of  the  topics  (such  as  current  mode  uses)  were  discussed  by  all  of 
them. 

The  IP  interviews  were  designed  to  be  partly  structured  and  partly  free¬ 
wheeling,  depending  upon  the  topic  areas  and  IP  insights  as  we  progressed.  The 
interviews  were  divided  into  four  parts.  First,  we  briefly  described  the  nature  of  the 
research  project  and  its  symbology  improvement  and  intelligent  moding  objectives. 
Next,  we  discussed  the  current  PNVS  moding  structure  in  detail,  its  advantages  and 
disadvantages,  typical  usage  patterns  during  a  mission,  and  the  potential  applica¬ 
tions  of  intelligent  moding. 

Subsequently,  we  began  to  explore  the  opportunities  for  advanced  HMD 
symbology  in  support  of  both  existing  task  requirements,  and  operational  tasks  as 
influenced  by  a  broad  range  of  new  technologies  expected  to  be  on  the  next  genera¬ 
tion  of  attack  helicopters.  These  discussions  were  subdivided  into  six  categories: 
flight  control  and  sensors,  navigation  and  low-level  flight,  voice  and  digital 
communications,  targeting  and  weapons  management,  aircraft  survival  equipment, 
and  mission  management. 

For  each  of  the  operational  tasks  and  new  MEPs  in  a  given  category,  we  clarified 
our  understanding  of  the  requirements  with  the  IPs  and  discussed  the  new  types  of 
symbols  to  potentially  aid  pilot  performance.  We  presented  a  variety  of  situations, 
mission  segments,  tasks,  and  information  elements,  confirming  our  understanding 
of  the  requirements,  and  suggesting  potential  new  types  of  symbols  to  aid  pilot 
performance. 

The  discussions  focused  primarily  upon  HMD  symbols  that  could  be  used  to  (a) 
identify  tactical  positions  on  the  ground  (conformally);  (b)  employ  in  communicat¬ 
ing  positions  to  the  CPG,  other  aircraft,  or  the  TOC;  (c)  permit  computer  systems, 
sensors,  or  other  aviators  to  alert  the  pilot  to  a  target,  obstacle,  or  other  position  in 
the  terrain;  or  (d)  stay  "head-up"  in  critical  situations.  For  each  potential  symbol 
type,  we  attempted  to  determine  the  specific  circumstances  of  its  utility;  its  degree  of 
improvement  over  current  capabilities;  and  the  most  appropriate  symbol  forms, 
locations,  and  behaviors.  For  high-payoff  symbols,  we  attempted  to  identify  intelli¬ 
gent  techniques  for  managing  the  symbol's  appearance  on  (or  removal  from)  the 
HMD. 

In  the  fourth  part  of  the  interview,  the  IPs  completed  a  brief  survey  of  current 
IHADSS  symbology.  The  survey  asked  the  participant  to  rate  the  usefulness, 
frequency  of  use,  and  design  quality  of  each  IHADSS  symbol. 
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Phase  I  Interview  Results — PRISMS  I  Utility 

PRISMS  I  was  installed  on  a  Macintosh  520c  laptop  computer  for  use  as  a 
knowledge  acquisition  device  during  interview  sessions.  The  twelve  IPs  were 
immediately  impressed  with  the  fidelity  of  the  PRISMS  I  symbology  portrayal,  espe¬ 
cially  with  the  symbol  animation  enabled,  and  several  asked  if  it  would  be  possible 
to  obtain  the  software  for  use  in  their  classroom  instruction.  They  explained  that  no 
part-task  simulator  existed  for  AH-64  symbology,  and  they  were  forced  to  simply 
draw  symbols  on  a  chalkboard  in  the  classroom. 

The  IPs  told  us  that  PRISMS  I,  with  a  set  of  appropriate  scenarios,  would  be 
very  valuable  in  teaching  symbology  interpretation  for  aircraft  activities.  In 
particular,  they  felt  PRISMS  I  would  aid  in  teaching  symbology  use  during  such 
flight  activities  as:  hover,  hovering  turn,  lateral  hover,  normal  takeoff,  rolling 
takeoff,  NOE  flight,  slope  operations,  traffic  pattern  flight,  normal  approach,  normal 
landing,  and  roll-on  landing. 

For  the  purposes  of  our  interviews,  PRISMS  I  served  initially  to  introduce  the 
Anacapa  staff  as  cognizant  of  IHADSS  symbology,  and  knowledgeable  in  attack 
helicopter  operations,  thus  facilitating  the  discussions.  During  the  course  of  the 
interviews,  and  as  anticipated,  PRISMS  I  provided  the  ability  for  the  IPs  and  inter¬ 
viewers  to  point  at  or  modify  the  behavior  of  the  symbols  on  the  screen,  yielding 
much  more  effective  knowledge  acquisition  than  would  be  otherwise  possible  based 
on  pilot  recall  and  description  of  symbol  behavior  and  pencil  sketches. 

Phase  I  Interview  Results — Opportunities  for  Intelligent  Symbology 
Management 

As  a  result  of  the  variety  of  methodologies  employed  in  the  Phase  I  effort,  liter¬ 
ally  dozens  of  opportunities  for  HMD  symbology  enhancements  and  intelligent 
symbology  management  were  identified.  Below  we  describe  just  a  few  of  these 
opportunities,  using  them  as  examples  of  the  many  types  of  valuable  improvements 
that  could  be  made. 

Examples  of  Findings:  Improvement  of  the  Existing  Mode  Structure 

We  quickly  learned  that  the  Apache  symbols  are  not  necessarily  used  as 
described  in  the  manuals  and  training  materials.  The  usage  patterns  are  often 
surprising,  and  helpful  in  interpreting  specific  information  requirements  and  their 
correlates.  For  example,  all  of  the  SMEs  indicated  that  the  Cruise  mode  is  rarely 
used  by  AH-64  pilots.  Only  " about  one  in  twenty  pilots  use  it  at  all,"  because  infor¬ 
mation  from  the  velocity  vector  (which  disappears  in  the  Cruise  mode)  is  needed  so 
frequently.  In  addition,  the  Bop-up  mode  is  very  rarely  used,  primarily  because  the 
current  Doppler  system  is  not  sufficiently  accurate  in  controlling  the  hover  box.  The 
hover  box  symbol  is  designed  to  mark  the  ground  position  under  the  aircraft  at  the 
initiation  of  a  bob-up  maneuver,  so  that  remasking  can  be  achieved  over  the  same 
(obstacle-free)  point.  The  navigation  system  inaccuracies,  however,  permit  the  box 
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to  drift  at  a  rate  of  up  to  21  feet  per  minute,  such  that  pilots  cannot  always  trust  it. 

As  a  result,  the  Transition  and  Hover  modes  are  used  most  of  the  time. 

One  of  the  reasons  the  Transition  mode  is  so  frequently  used  is  that  the  veloc¬ 
ity  vector  can  provide  the  necessary  crab  angle  for  straight-line  flight  in  a  cross-wind 
without  any  computations  by  the  pilot.  Although  tracking  the  command  heading 
symbol  will  lead  to  the  next  waypoint,  following  the  command  heading  in  a  cross- 
wind  will  result  in  a  curved  flight  path.  To  fly  a  straight  path  to  the  waypoint,  it  is 
only  necessary  to  orient  the  aircraft  so  that  the  command  heading  symbol  (Doppler 
caret)  is  positioned  half  way  between  the  lubber  line  and  the  velocity  vector,  as 
shown  in  Figure  2-6.  This  capability  is  often  useful  in  traffic  patterns  as  well.  In 
applications  such  as  this,  it  does  not  matter  that  the  Transition  mode  velocity  vector 
is  "saturated"  (reaches  the  edge  of  the  display  area)  at  speeds  above  60  knots. 


Figure  2-6.  The  Transition  mode  used  to  fly  a  straight  path  in  a  cross-wind. 

In  applications  with  slower  speeds,  such  as  during  aircraft  turns,  the  propor¬ 
tional  length  of  the  velocity  vector  in  the  Transition  mode  can  be  used  as  an  imme¬ 
diate  cue  for  adjusting  speed:  for  example,  two-thirds  of  the  way  to  the  edge  of  the 
screen  is  40  knots,  groundspeed  (as  opposed  to  the  true  airspeed  digital  readout).  Yet 
another  innovative  use  for  the  velocity  vector  is  as  an  indicant  of  whether  the 
aircraft  is  moving  through  terrain  in  NOE  flight  with  a  good  nose-tail  trim  "so  that 
you  don't  hit  objects  with  the  tail  as  you  go  by." 

The  Transition  mode  and  Hover  mode  symbology  differ  in  only  two  ways:  the 
Transition  mode  adds  a  horizon  line,  and  changes  the  velocity  vector  saturation 
point  from  6  to  60  knots.  The  Hover  mode's  more  sensitive  velocity  vector  (and 
acceleration  cue)  are  valuable  for  precise  hovering  both  at  airfields  and  in  combat 
areas.  Yet,  the  horizon  line  provides  important  information  not  only  during 
contour  and  NOE  flight,  but  for  takeoff  and  landing  at  night.  At  takeoff,  the  pitch 
information  from  the  horizon  line  is  particularly  valuable,  and  on  landing,  the 
"wings-level"  information  is  necessary.  On  a  normal  landing,  the  Hover  mode  is 
used  until  the  main  gear  touches  the  ground,  when  the  pilot  switches  to  the  Transi¬ 
tion  mode  to  view  the  horizon  line.  This  is  of  particular  importance  in  a  slope 
landing.  The  pilot  is  very  busy  at  this  point  and  fumbling  for  a  manual  mode 
switch  is  a  burden. 
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Intelligent  Symbology  Management.  Nearly  all  of  the  typical  pilot's  symbology 
changes  are  between  the  Hover  and  Transition  modes.  Many  of  the  IPs  indicated 
that  intelligent  symbology  management  could  be  applied  to  making  this  change, 
particularly  since  the  change  often  occurs  during  periods  of  heaviest  pilot  workloads 
when  thumb  fumbling  and  switch  direction  confusions  are  the  most  prevalent  and 
frustrating. 

The  IPs  offered  a  variety  of  strategies  for  implementing  this  intelligent  moding. 
Several  indicated  that  it  would  be  possible  to  simply  change  from  the  Hover  to  the 
Transition  mode  based  on  a  groundspeed  above  6  knots,  when  the  Hover  mode 
velocity  vector  saturates.  On  the  other  hand,  as  some  IPs  pointed  out,  the  fact  that 
the  vector  is  saturated  can  itself  be  important  information.  Thus,  others  suggested 
making  the  change  at  15,  20,  28,  or  even  32  knots.  Additional  study  will  be  required 
to  define  an  acceptable  speed  threshold. 

Other  IPs  suggested  more  sophisticated  approaches  than  using  only  ground- 
speed  as  the  cue  for  the  mode  change.  Such  approaches  included  detecting  a  change 
in  power  setting  to  10%  above  a  baseline  hover  power  (which  changes  from  day  to 
day),  a  change  of  3°  in  downward  pitch,  or  extreme  pitch  or  bank  changes  to  trigger 
the  onset  of  the  Transition  mode.  Several  IPs  noted  that  the  "squat  switch"  in  the 
landing  gear  might  be  used  to  initiate  the  mode  change,  if  it  is  sensitive  enough  to 
detect  touchdown  (instead  of  full  weight)  on  the  gear;  otherwise,  another  switch 
could  be  added  to  detect  the  initial  contact  of  the  landing  gear. 

Because  only  the  horizon  line  and  type  of  velocity  vector  change  between  the 
two  modes,  it  is  certainly  reasonable  to  consider  intelligent  presentation  of  these 
individual  symbols,  dispensing  with  mode  switching  altogether.  There  are  also 
some  reasons  for  considering  a  manual  (or  voice)  override  capability  as  a  pilot's 
option.  For  example,  several  IPs  described  the  temporary  selection  of  Transition 
mode  during  hover  maneuvers  to  obtain  a  less-sensitive  velocity  vector  for  settling 
out  aircraft  attitude  difficulties  before  returning  to  the  Hover  mode  with  its  very 
sensitive  velocity  vector.  On  the  other  hand,  rules  could  be  developed  employing 
attitude  volatility  to  select  the  6-knot  or  60-knot  velocity  vector.  As  described  below, 
though,  such  automation  demands  that  effective  cues  be  provided  to  prevent 
symbol  confusions. 

Examples  of  Findings:  Avoiding  Symbol  Confusions 

One  of  the  recurring  difficulties  in  designing  an  intelligent  symbology 
management  system  is  ensuring  that  its  automated  changes  do  not  confuse  the 
pilot.  In  general,  symbol  confusions  among  the  current  AH-64  symbol  set  are 
unusual.  Although  it  is  difficult  for  students  to  learn  the  use  of  the  velocity  vector 
and  acceleration  cue,  the  difficulty  is  based  on  the  demanding  nature  of  the  task,  not 
design  problems  with  the  symbols  themselves. 

The  most  critical  confusions  regarding  symbol  meanings  in  the  AH-64  result 
from  failure  to  recognize  the  current  mode.  The  similarity  of  the  four  modes  is 
shown  in  Figure  2-7.  For  example,  the  Hover  and  Transition  modes  are  sometimes 
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confused  (although  the  Transition  mode  includes  the  horizon  line).  Mistaking  a  6- 
knot  velocity  vector  for  a  60-knot  velocity  vector  (or  vice  versa)  could  well  lead  to 
inappropriate  control  inputs.  One  IP  described  a  near-accident  of  this  type  using  the 
current  manual  moding,  where  he  mistook  a  20-knot  right-rear  drift  for  a  2-knot 
movement.  He  strongly  agreed  with  our  suggestion  that  the  two  velocity  vectors  be 
shaped  differently  so  that  the  6-knot  and  60-knot  versions  would  be  instantly 
discriminable.  The  two  vectors  could  differ  in  thickness,  continuity  (solid  versus 
dashed),  or  in  the  presence  of  a  symbol  at  the  end  of  the  vector.  This  type  of  coding 
would  provide  especially  important  feedback  in  the  case  of  intelligent  symbology 
moding  of  the  Hover  and  Transition  modes,  so  that  the  pilot  would  be  immediately 
aware  that  the  mode  shift  had  taken  place. 


Hover  Mode 


W  30  33  N  3  6  E 


CQ 


Cruise  Mode 


Transition  Mode 


Bob-Up  Mode 


Figure  2-7.  The  four  IHADSS  mode  screens:  Hover  mode,  Transition  mode,  Cruise 
mode,  and  Bob-up  mode. 

Another  potentially  dangerous  confusion  among  modes  pertains  to  the  Bob-up 
mode.  When  this  mode  is  selected,  the  command  heading  symbol  is  locked  to  the 
aircraft  heading,  so  that  the  pilot  can  remask  the  aircraft  with  the  tail  in  the  same 
direction  as  when  the  bob-up  was  begun.  If  the  pilot  forgets  to  deselect  the  Bob-up 
mode,  and  subsequently  mistakes  the  command  heading  symbol  for  the  next 
waypoint,  "he  could  fly  directly  toward  the  enemy  target  he  had  been  observing." 
These  confusions  could  be  avoided  if  the  command  heading  caret  was  replaced  by  a 
different  symbol  (e.g.,  a  dotted  oval)  in  the  Bob-up  mode. 

Yet  another  type  of  confusion,  especially  for  students,  stems  from  the  off-axis 
viewing  of  the  screen-fixed  information.  Symbology  designed  to  be  used  along  the 
aircraft  axis  may  be  spatially  incompatible  when  the  pilot  turns  his  head  to  the  left 
or  right.  For  example,  the  velocity  vector  does  not  seem  to  move  in  the  correct 
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direction  when  the  pilot  is  looking  off-axis.  Nevertheless,  the  cure  for  these 
problems  is  not  obvious.  It  could  be  "very  disruptive  and  dangerous"  if  these 
symbols  simply  disappeared  or  changed  radically  when  the  pilot  is  looking  off-axis. 
For  example,  "you  need  the  velocity  vector  when  you  look  to  the  side  for  lateral 
hover  and  for  correcting  fore  and  aft  drift."  These  kinds  of  confusions,  based  on 
spatial  incompatibilities  of  display  and  response  pairs,  are  unlikely  to  be  resolved  or 
mitigated  without  insightful  experimentation,  such  as  that  of  Haworth  &  Seery 
(1992),  and  are  at  the  core  of  any  inventory  of  HMD  research  issues.  Interview 
techniques  alone  cannot  address  these  complex  spatial  issues. 

Examples  of  Findings:  Symbol  Deletions  and  Additions 

In  order  to  best  address  the  issue  of  decluttering  the  existing  symbology  presen¬ 
tation,  the  interviews  were  initially  structured  to  identify  the  particular  situational 
attributes  that  would  lead  to  the  removal  of  a  symbol  on  the  screen.  In  addition  to  a 
detailed  discussion  about  the  utility  of  every  current  symbol,  7-point  rating  scales 
were  administered  to  the  IPs  to  judge  the  usefulness,  frequency  of  use,  and  design 
quality  of  every  symbol. 

In  general,  although  the  SMEs  are  very  concerned  about  HMD  clutter,  almost 
none  of  the  symbols  were  seen  as  "less  than  valuable."  Only  the  radar  altitude 
vertical  tape  and  scale  were  clearly  deemed  potential  candidates  for  removal.  This 
was  because  they  "take  up  so  much  screen  space,"  the  tape  movements  can  be  "a 
distraction,"  and  because  the  radar  altitude  digital  readout  combined  with  the 
vertical  speed  scale  are  believed  to  provide  more  relevant  information  in  a  more 
directly  useful  form. 

On  reviewing  the  utility  of  each  of  the  existing  symbols  in  detail  with  each  of 
the  IPs,  we  found  that  removal  of  any  of  the  existing  flight  symbology  was  unlikely 
to  have  a  high  payoff.  Instead,  we  concluded  that  more  could  be  learned  by  identify¬ 
ing  the  situations  in  which  a  new  symbol  might  be  temporarily  helpful.  Thus,  the 
components  of  mission  activities  previously  identified,  including  segments, 
functions,  and  tasks,  and  their  required  information  elements,  were  used  to  explore 
symbol  requirements  in  the  interviews,  in  order  to  identify  activities  in  which  new 
symbols  would  clearly  be  useful  or  clearly  not  useful  and  to  determine  methods  for 
intelligent  symbology  management.  Suggested  additional  symbols  generally  fell 
into  one  of  four  categories:  (a)  information  enhancement,  (b)  cautionary,  (c)  control 
setting,  and  (d)  spatial/  geographic. 

New  Information  Enhancement  Symbols:  Wind  Data 

In  certain  cases,  information  is  readily  available  from  the  aircraft  systems  that 
would  be  useful  to  the  pilot,  but  it  is  not  presented  because  situational  rules  for  its 
utility  have  not  been  defined.  An  excellent  example  is  that  of  wind  data.  The 
Apache  true  airspeed  (TAS)  readout  does  not  currently  show  the  relative  direction 
of  the  wind,  although  it  does  show  a  speed  even  if  the  helicopter  is  not  moving. 
Wind  direction  data  would  be  useful  on  the  HMD,  especially  in  the  AH-64,  because 
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"you  can't  really  feel  it  [wind],  even  in  a  30-knot  cross  wind  because  the  Digital 
Automated  Stabilization  Equipment  (DASE)  is  so  effective."  Knowledge  of  the  wind 
direction  and  velocity  might  be  moderately  helpful  in  normal  flight,  although  the 
Velocity  Vector  cues  are  usually  sufficient. 

In  other  situations,  however,  "wind  can  get  you  in  trouble,  such  as  in  the  cases 
of  cranking  or  shutting  down,  takeoff  with  a  tail  wind,  or  descending  into  trees  with 
a  tail  wind  that  might  push  you  into  trouble."  Additional  needs  for  wind  data 
identified  by  the  IPs  included  "rocket  engagements,"  "hover  taxi,"  "restricted  area 
landings,"  "bob-ups,"  "landing  at  unfamiliar  fields,"  and  "hovering  at  night  in  the 
trees."  Wind  data  are  needed  most  of  all  to  indicate  head  winds  and  tail  winds, 
which  are  least  detectable  by  the  pilot  (no  aircraft  attitude  changes  are  available  as  a 
cue).  Flying  into  the  wind  is  also  an  important  factor  in  aircraft  recovery 
maneuvers,  such  as  after  loss  of  an  engine. 

The  IPs  suggested  that  a  small  symbol  be  placed  near  or  below  the  airspeed 
symbol,  as  shown  in  Figure  2-8,  to  show  the  computed  wind  speed  and  its  direction 
when  in  the  Hover  or  Bob-up  modes,  when  it  is  most  important.  Other  pilots 
suggested  that  the  indicator  be  placed  in  the  lower-left  corner  of  the  screen.  Several 
thought  a  numeric  readout  of  wind  velocity  should  also  be  included  with  the 
relative  direction  indicator.  The  pointer  should  "probably  be  damped"  to  prevent 
erratic  behavior  if  winds  are  gusting. 


Figure  2-8.  Candidate  information  enhancement  symbol  for  wind  direction  (Hover 
mode). 

Intelligent  Symbology  Management.  There  are  several  ways  to  use  intelligent 
symbology  management  to  cause  a  wind  symbol  to  appear.  For  example,  one  IP 
suggested  that  it  could  be  enabled  "when  you  get  to  the  point  of  change  from 
positive  to  negative  airflow,  and  aircraft  speed  is  less  than  40  knots."  Other  intelli¬ 
gent  moding  cues  could  be  developed  based  on  specific  wind  speeds,  relative  direc¬ 
tions,  and  current  aircraft  altitudes  and  airspeeds.  A  verbal  command  to  display  this 
data  for  a  few  seconds  would  also  be  acceptable  to  most  SMEs. 
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New  Information  Enhancement  Symbols:  Ground  Speed  Display 

The  IPs  recommended  that  a  ground  speed  display  be  provided  to  improve 
accuracy  in  determining  expected  time  of  arrival  (ETA)  at  various  waypoints.  These 
times  may  be  critical  to  mission  success  and  aircraft  survival.  For  example,  inbound 
and  outbound  Passage  Point  arrival  times  must  be  very  accurate  (±  30  seconds  accu¬ 
racy  is  allowable)  to  avoid  risking  "friendly  fire."  The  existing  TAS  display  is,  by 
definition,  influenced  by  winds  and  does  not  provide  ground  speed  unless  the  pilot 
"fails"  the  ADS.  As  an  alternative,  the  CPG  can  be  further  burdened  with  a  request 
to  repeatedly  calculate  GS  and  tell  the  pilot  to  "speed  up  or  slow  down  a  little." 

Given  the  current  speed,  the  Doppler  system  alone  can  be  used  to  determine  the 
ETA  at  a  waypoint.  However,  the  system  does  not  "know"  whether  the  aircraft  is  on 
time,  or  by  how  much  it  is  ahead  or  behind  schedule. 

Some  kind  of  a  "speed  bug"  would  be  very  helpful  in  these  cases.  It  should  also 
be  possible  to  select  a  point  other  than  the  next  waypoint  for  the  calculation,  such  as 
a  Passage  Point  several  waypoints  removed,  and  still  calculate  using  the  planned 
flight  path  (not  a  path  directly  to  the  Passage  Point).  A  related  aid  would  be  a  "time 
to  leave"  indicator  to  cue  the  pilot  regarding  when  to  depart  holding  areas,  assembly 
areas,  or  other  positions.  Time  should  be  shown  in  minutes  and  tenths  of  minutes. 

Intelligent  Symbology  Management.  The  ground  speed  symbol  could  be  moded 
to  appear  only  when  in  flight  between  waypoints,  rather  than  in  hover  or  bob-up 
activities.  Its  appearance  and  behavior  would  be  a  joint  function  of  the  flight  plan 
"currently"  in  effect,  clock  time,  and  ground  speed.  It  could  be  used  to  present  a 
"time  remaining"  display  in  minutes  and  tenths  of  minutes  or,  if  straight-line  flight 
is  anticipated,  the  symbology  could  be  presented  as  an  airspeed  requirement  to 
compare  to  the  TAS  reading.  The  appearance  and  behavior  of  a  "time  to  leave" 
indicator  would  also  be  a  function  of  the  flight  plan  currently  in  effect,  clock  time, 
and  planned  ground  speed  to  subsequent  positions. 

New  Information  Enhancement  Symbols:  Planned  Track  Display 

Although  straight  lines  are  drawn  between  the  waypoints  on  the  pilot's 
kneeboard  map  during  route  planning,  it  is  seldom  expected  that  the  actual  flight 
path  will  follow  straight  line,  especially  when  terrain  masking  is  desirable.  Never¬ 
theless,  in  order  to  deconflict  the  aircraft  flight  path  with  other  friendly  units  and 
artillery  fire  zones,  the  S-3  may  define  a  maximum  distance  from  the  straight  line  as 
a  flight  corridor  of  a  given  width. 

The  pilot  often  wants  to  know  the  distance  of,  and  direction  to,  the  straight  line 
between  two  waypoints,  so  that  he  can  quickly  return  to  the  planned  leg  and  use  the 
originally  planned  headings  and  speeds.  If  a  digital  map  display  were  available,  the 
solution  would  be  obvious.  But  with  the  current  map,  such  navigation  issues  are 
burdensome  for  both  pilot  and  CPG.  Currently,  there  are  two  ways  to  obtain  this 
information:  (a)  the  CPG  must  accurately  identify  the  aircraft  location  and  calculate 
the  distance  and  direction  for  the  pilot,  or  (b)  the  pilot  must  recall  or  otherwise 
retrieve  the  leg  bearing  and  fly  in  the  approximate  direction  of  the  line  until  the 
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Command  Heading  Caret  lines  up  with  that  bearing  on  the  Heading  Scale.  The 
SMEs  were  unable  to  offer  a  display  format  or  location  without  more  detailed 
consideration  of  all  the  implications. 

Intelligent  Symbology  Management.  Information  is  available  from  the  naviga¬ 
tion  system  to  compute  the  relationship  for  the  planned  track,  cross  track  angle,  and 
the  current  aircraft  position.  This  information  could  be  moded  to  appear  at  a  certain 
distance  from  the  leg  line,  depending  upon  an  acceptable  flight  corridor  width,  an 
ETA  computation,  or  a  verbal  command  from  the  pilot. 

New  Information  Enhancement  Symbols:  Target  Hand-Off  Symbol 

The  IPs  described  three  methods  currently  used  to  perform  target  hand-offs: 
voice  with  directions,  voice  with  coordinates,  and  laser  tracking.  In  the  first 
method,  the  target  range  and  bearing  from  one  viewer  is  transmitted  to  the  second 
aircraft  and  the  range  and  bearing  from  the  second  aircraft  is  estimated  based  upon 
the  relative  position  of  the  two  friendly  aircraft.  In  the  second  method,  the  CPG 
lases  the  target  and  the  flight  control  computer  (FCC)  provides  8-digit  grid  coordi¬ 
nates  which  are  transmitted  by  voice  radio.  The  recipient  of  the  coordinates  enters 
them  in  the  DEK  and  slaves  the  TADS  to  the  coordinates.  In  the  third  and  most 
accurate  method,  the  CPG  lases  the  target  while  the  CPG  in  the  second  aircraft 
searches  for,  then  tracks,  the  laser  energy  with  the  laser  spot  tracker.  For  each 
method,  a  price  is  paid  in  terms  of  performance  time,  potential  detection  by  the 
enemy,  or  both. 

Target  hand-off  could  be  improved  with  a  "passive  ranging"  capability  made 
possible  through  greatly  improved  head-tracker  accuracy  and  the  incorporation  of 
digital  terrain  elevation  data  bases.  The  computation  method  is  depicted  in  Figure 
2-9,  below.  The  computed  coordinates  could  then  be  sent  verbally,  or  by  "data-burst" 
in  which  a  conformal  target  marker  is  presented  in  the  recipient's  HMD  symbology. 


Figure  2-9.  The  geometry  of  passive  ranging  with  the  HMD. 

A  passive  ranging  capability  with  a  cued  LOS  symbol  would  eliminate  the 
possibility  of  detection  of  laser  energy  by  the  enemy,  reduce  the  risk  of  detecting 
voice  traffic,  and  speed  the  target  hand-off  process  significantly.  In  addition,  as  the 
IPs  pointed  out,  it  would  prevent  the  inadvertent  lasing  of  friendly  troops.  As  well, 
laser  returns  are  often  lost  due  to  smoke  and  terrain  obstruction.  Although  the  pilot 
could  send  or  receive  targets  in  this  manner,  in  most  cases  this  is  the  domain  of  the 
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CPG  because  the  magnified  optics  of  the  TADS  permit  more  distant  target  identifica¬ 
tion  and  acquisition.  However,  the  pilot  may  well  use  the  gun  for  close-in  targets, 
and  passive  ranging  with  the  LOS  reticle  would  immediately  provide  the  range  data 
necessary  for  the  FCC  to  accurately  compute  the  instantaneous  impact  point  of  the 
rounds.  Currently  he  must  either  ask  the  CPG  to  lase  the  target  for  distance  or 
"walk"  the  rounds  onto  the  target  to  correct  the  error. 

Intelligent  Symbology  Management.  The  target  hand-off  symbol  could  be  set  to 
appear  automatically  in  the  recipient's  view,  if  data-burst  to  his  aircraft.  For  the 
pilot's  gun  solutions,  the  passive  range-finding  and  data  entry  in  the  FCC  could  be 
performed  automatically  when  the  pilot  actions  the  gun  and  places  the  LOS  reticle 
over  the  target.  A  visible  change  in  the  reticle  could  be  used  to  indicate  that  the 
ranging  solution  had  been  achieved. 

New  Cautionary  Symbols:  Aircraft  Control  Settings 

In  general,  panel-mounted  advisories  are  acceptable  to  the  pilots.  However,  in 
certain  situations  they  may  not  be  seen,  and  it  may  be  more  effective  to  present  the 
caution  message  within  the  field  of  view  (and  perhaps  add  voice  warnings).  One 
such  case  is  that  of  the  parking  brake  setting.  The  physical  position  of  the  small 
parking  brake  handle  (in  front  of  and  below  the  right  knee)  is  the  only  cue  to  its 
status.  The  parking  brake  must  be  set  to  "on"  for  landings  in  unpaved  terrain,  but 
"it  is  easy  to  forget  and  leave  it  on  when  you  get  to  a  hard  field."  A  roll-on  landing 
onto  a  hard  field  will  blow  out  one  or  both  tires,  possibly  causing  swerving  and 
aircraft  rollover. 

The  tail  wheel  lock  setting  is  a  similar  case,  although  not  as  critical  as  the 
parking  brake  setting.  The  tail  wheel  can  be  locked  to  permit  fore-aft  wheel  motion 
only.  If  the  tail  wheel  is  unlocked,  it  can  cause  fish-tailing  during  a  roll-on  landing. 
There  is  a  warning  light  on  the  switch  if  it  is  unlocked,  but  it  is  hard  to  see  under  the 
glareshield  at  the  left  side  of  the  instrument  panel  in  the  pilot's  station. 

Another  useful  advisory  that  could  be  added  is  a  "DASE  Off"  message.  The 
DASE  facilitates  control  of  pitch,  roll,  and  yaw  axes  and  should  always  be  turned  on 
before  moving  the  aircraft.  A  caution  light  exists,  but  is  usually  too  dim  to  be 
noticed  in  its  current  position.  Although  it  is  possible  to  fly  the  aircraft  without  the 
DASE,  the  aircraft  is  "very  squirrelly"  and  could  be  dangerous,  especially  in  forma¬ 
tion. 


Intelligent  Symbology  Management.  An  intelligent  symbology  management 
system  would  be  extremely  useful  if  it  could  detect  situations  in  which  such  normal 
settings  suddenly  become  dangerous,  and  display  appropriate  cautions  to  the  pilot. 
Although  the  control  settings  could  easily  be  sensed,  the  IPs  were  not  certain  how  to 
best  detect  that  a  roll-on  landing  was  in  progress.  It  is  probable  that  the  message 
should  appear  at  low  altitudes  (estimates  ranged  from  5  to  50  feet)  and  at  ground 
speeds  appropriate  for  a  roll-on  landing  (about  30  knots),  but  additional  cues  should 
be  identified.  An  intelligent  moding  of  the  DASE  alert  could  be  based  on  the  "squat 
switch"  which  detects  weight  off  the  main  landing  gear. 
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New  Cautionary  Symbols:  Obstacle  Avoidance 

Obstacles  to  flight  should  be  a  top  priority  for  HMD  display.  The  dangers  of 
wire  strikes  and  the  difficulty  in  detecting  wires  at  night  are  well  known.  Some 
areas  of  the  world,  such  as  Germany,  are  particularly  dangerous  due  to  the  number 
of  wires  even  at  high  altitudes.  During  planning,  wires  and  other  known  obstacles 
are  marked  on  maps  for  CPG  reference.  In  the  future,  preplanned  obstacles  such  as 
towers  and  wires  will  be  stored  with  a  variety  of  mission  management  data  carried 
to  the  aircraft  in  digital  form.  New  obstacle  detection  technologies  are  under  devel¬ 
opment  that  can  identify  additional  unmarked  wires,  poles,  and  towers  in  real  time 
onboard  the  aircraft.  To  the  extent  possible,  pilots  want  the  symbology  to  "look  like" 
the  type  of  obstacle  depicted,  and  appear  in  the  field  of  view  in  the  appropriate,  real- 
world  position  so  that  the  most  appropriate  actions  can  be  taken. 

Intelligent  Symbology  Management.  Symbols  to  identify  both  stored  and 
detected  obstacles  can  be  intelligently  moded  to  appear  to  the  pilot  when  the  aircraft 
is  at  some  given  distance  from  the  obstacle,  and  moving  at  a  speed  and  in  a  heading 
likely  to  result  in  obstacle  contact  within  brief  periods  of  time  unless  evasive 
maneuvers  are  initiated. 

New  Control  Setting  Symbols:  Radio  Frequency  Numerics 

In  many  cases,  temporary  symbols  or  alphanumerics  could  be  presented  on  the 
HMD  to  aid  the  pilot  in  control  settings  whose  results  are  not  otherwise  obvious. 
Detection  of  the  control  actuation  makes  intelligent  symbology  management  easy  to 
achieve  in  this  domain.  Radio  frequencies  are  one  of  the  best  examples  of  this 
category.  In  general,  the  pilot  (rather  than  the  CPG)  is  the  one  to  make  the  radio 
calls.  The  pilot  may  need  to  change  frequencies  to  contact  a  ground  commander  or 
another  unit  and,  if  the  CPG  is  busy,  the  pilot  must  also  do  this  himself.  To  change 
frequencies,  the  pilot  must  presently  take  his  left  hand  off  the  collective  to  hold  the 
cyclic,  while  twisting  his  body  around  to  the  right  and  using  his  right  hand  to  turn 
the  frequency  selector  knobs.  The  Army  has  not  made  any  policy  statements  about 
this  being  an  "acceptable"  procedure. 

In  addition  to  this  distracting  contortion,  it  is  difficult  to  read  the  frequency 
display,  and  may  actually  be  impossible  in  mission-oriented  protective  posture 
(MOPP)  gear.  If  the  pilot  could  read  the  radio  frequency  on  the  HMD,  he  could  at 
least  keep  his  head  up,  even  if  the  manual  entry  remained  difficult.  Changing 
transponder  settings  is  similar  to  changing  radio  frequencies,  except  that  the  current 
displays  are  even  more  difficult  to  read.  The  use  of  a  voice  input  procedure  "would 
solve  a  lot  of  problems,"  but  the  pilot  would  still  want  to  see  the  selected  frequency 
for  confirmation. 

Intelligent  Symbology  Management.  The  activation  of  a  radio  frequency  or 
transponder  setting  control  could  easily  be  used  as  the  cue  to  present  the  appropriate 
frequency  information  in  a  head-up  manner.  The  most  likely  area  for  display  would 
be  in  the  High- Action  Display  (HAD)  at  the  bottom  of  the  IHADSS  field  of  view. 
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New  Spatial/Geographic  Symbols:  Situation  Awareness  Cueing 

Much  of  the  information  that  an  Apache  pilot  must  process  is  related  to 
friendly  and  enemy  positions  in  space  or  on  the  ground.  The  current  methods  of 
correlating  these  positions  with  directions  from  the  aircraft  or  specific  sites  on  the 
earth  are  among  the  most  difficult,  error-prone,  and  time-consuming  tasks  the  pilot 
must  perform.  Visual  cueing  to  positions  in  the  terrain  is  presently  rather  clumsy 
in  the  AH-64,  even  though  much  improved  over  the  AH-1  Cobra  aircraft.  The 
procedure  now  used  in  the  AH-64A  to  cue  the  pilot  to  a  position  in  the  terrain  is  as 
follows: 

(1)  If  the  positions  are  preplanned  and  entered  in  the  FCC,  the  CPG  will  select 
them  with  the  Data  Entry  Keyboard  (DEK)  and  slave  the  TADS  to  them. 

(2)  To  point  out  a  position  to  the  pilot  that  is  not  pre-entered  in  the  FCC,  the 
CPG  must  enter  the  8-digit  grid  coordinates  in  the  DEK  or  find  the  posi¬ 
tion  by  visual  search  and  fire  the  laser  to  obtain  coordinates.  He  then 
slaves  the  TADS  to  the  position. 

(3)  The  pilot  can  then  set  the  ACQ  SEL  control  to  "CPG"  so  that  the  cued  LOS 
reticle  is  the  azimuth /elevation  of  the  coordinate's  location  in  the  terrain, 
and  use  the  cued  LOS  dots  to  acquire  the  cued  LOS  reticle.  This  cue,  of 
course,  is  temporary,  lasting  only  until  the  TADS  LOS  is  redirected.  It  is  at 
this  point  that  the  pilot  is  truly  able  to  "look  where  the  CPG  is  looking." 

While  this  cueing  procedure  is  effective,  it  demands  that  the  CPG  and  his 
TADS  be  fully  dedicated  to  support  maintaining  the  pilot's  situation  awareness.  In 
fact,  the  TADS  is  primarily  in  use  for  target  acquisition  and  the  CPG  is  heavily 
burdened  with  navigation,  communication,  and  weapons  handling  tasks.  There  are 
many  potential  examples  of  spatial  /geographic  cueing  to  aid  Apache  helicopter 
pilots  who  "want  to  know  where  everybody  is,  friendly  and  enemy,  and  stay  out  of 
everyone's  radar." 

When  a  digital  map  becomes  available  in  the  cockpit,  the  pilot's  designation  of 
any  map  position  could  result  in  the  appearance  of  a  "pointer"  symbol  identifying 
that  position  in  the  real-world  terrain.  In  a  similar  manner,  the  pilot's  LOS  could  be 
used  to  identify  a  feature  in  the  real  world  and  enter  a  symbol  on  the  digital  map  for 
immediate  or  later  reference.  Thus,  the  difficult  problems  of  map-terrain  correla¬ 
tion  would  be  overcome.  Even  in  the  near  term,  without  a  digital  map,  it  should  be 
possible  for  the  pilot  to  select  any  preplanned  position  for  "virtual  cueing,"  without 
involving  the  TADS  or  the  CPG,  as  long  as  the  head  tracker  and  navigation  system 
accuracy  are  improved.  The  next  sections  discuss  some  specific  examples  of  situa¬ 
tion  awareness  cueing — conformal  waypoint  markers,  engagement  area  depiction, 
target  position  cues,  mission  management  data,  and  aircraft  survivability 
equipment  (ASE)  cues. 
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New  Spatial/Geographic  Symbols:  Conformal  Waypoint  Indicators 

The  Apache  pilot  is  almost  entirely  dependent  on  the  CPG  for  navigation 
information  except  for  the  Command  Heading  Caret.  As  a  result,  the  pilot's  situa¬ 
tion  awareness  is  often  less  than  optimal.  Although  he  carries  a  simple  strip  map  of 
the  planned  route  on  a  knee  board,  he  is  usually  unable  to  read  it  at  night,  and  may 
try  to  memorize  the  flight  legs  (e.g.,  the  heading  to  each  new  waypoint).  The  CPG 
must  help  the  pilot  by  "talking  him  through"  the  waypoints.  One  of  the  highest 
payoffs  for  intelligent  Symbology  management  would  be  to  use  the  HMD  to  auto¬ 
matically  display  conformal  markers  for  real-world  waypoints.  This  technological 
capability  has  already  been  demonstrated  by  the  RAH-66  Comanche  "lollipops"  that 
unambiguously  indicate  the  next  two  waypoints  to  the  pilot,  shown  in  Figure  2-10. 


Figure  2-10.  Conformal  waypoint  ("lollipop")  symbols  adapted  from  the  RAH-66. 

Conformal  Waypoint  Indicators  were  viewed  by  all  the  IPs  in  our  survey  as  a 
valuable  aid  to  visualization  of  the  route  in  the  real  world  terrain,  reducing  the 
requirements  for  guidance  by  the  CPG,  and  reducing  workload  for  both  crewmem¬ 
bers.  The  pilots  suggested  that  similar  markers  could  be  used  for  Assembly  Areas, 
Holding  Areas,  FARP  sites,  and  other  friendly  positions.  A  simple  ball  with  the 
waypoint  number  and  stem  was  judged  to  be  an  acceptable  format.  No  specific 
shapes  are  required  to  encode  other  types  of  points  because  "everybody  knows  that 
number  10  is  the  holding  area." 

Intelligent  Symbology  Management.  Intelligent  moding  of  these  markers 
could  be  controlled  by  current  aircraft  position  with  respect  to  planned  waypoints.  A 
voice  command  system  should  also  be  available  to  show  the  direction  of  any  of  the 
planned  positions  at  any  time. 
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New  Spatial/Geographic  Symbols:  Engagement  Area  (EA)  Depiction 

An  EA  is  planned  in  advance,  but  even  if  it  does  not  change  enroute,  locating 
its  position  in  the  dark  can  be  a  problem.  In  addition,  the  lead  aircraft  in  an  attack 
mission  designates  firing  sectors  to  prevent  expending  multiple  weapons  on  a  single 
target,  and  radios  these  sectors  to  the  other  aircraft.  Sector  boundaries  are  often  hard 
to  identify  in  the  dark,  especially  without  prominent  landmarks  to  divide  the  EA. 

The  problem  is  made  even  more  difficult  if  the  mission  has  been  changed  enroute, 
with  new  EA  positions  and  boundaries  assigned.  Target  engagement  orders  could  be 
greatly  simplified  by  a  conformal  depiction  of  the  EA.  This  would  help  the  pilot 
orient  the  aircraft  appropriately,  obtain  masking,  and  prepare  for  engagement.  The 
IPs  state  that  both  the  pilot  and  the  CPG  should  be  able  to  see  the  conformal  sector 
markers.  Some  pilots  suggested  that  color  coding  be  used  to  show  specific  firing 
sectors  and  no-fire  zones. 

Intelligent  Symbology  Management.  Intelligent  moding  of  the  EA  data  presen¬ 
tation  would  occur  with  aircraft  approach  to  the  BP  area.  Sector  data  would  be  added 
upon  transmission  of  a  digital  message  from  the  lead  aircraft. 

New  Spatial/Geographic  Symbols:  Target  Position  Cue 

A  target  position  indicator  is  sorely  needed  to  supplement  the  PNVS  symbol 
set.  There  is  currently  no  way  for  the  pilot  to  determine  the  location  of  targets 
without  receiving  directions  from  the  CPG.  The  CPG  can  slave  the  TADS  to  an 
offset  target  and  get  continuous  range  and  bearing  data,  but  must  continuously  relay 
the  data  to  the  pilot  by  intercom,  interrupting  his  target  searching  tasks.  The  pilot's 
Command  Heading  Caret  is  driven  only  by  waypoints,  not  by  targets.  Waypoints  are 
entered  in  the  Doppler  and  targets  are  entered  in  the  DEK.  Unfortunately,  the  two 
systems  do  not  share  coordinate  data.  The  pilot  has  cues  neither  for  target  direction 
nor  target  distance  and  he  can  become  disoriented  with  respect  to  his  relative  orien¬ 
tation  toward  the  target.  As  a  result,  he  may  mistakenly  select  routes  or  positions  on 
the  unmasked  side  of  landforms.  Target  distances  are  important  in  selecting 
countermeasures  and  firing  positions. 

Conformal  target  position  cues  would  greatly  help  in  initial  orientation  of  the 
aircraft,  thus  reducing  TADS  search  times.  Target  cues  could  be  provided  to  the 
pilot  as  small  indicants  in  the  Heading  Scale  when  at  long  distances,  and  as  confor¬ 
mal  cues  to  a  specific  location  in  the  terrain  when  approaching  more  closely.  The 
IPs  suggested  that  symbols  should  be  kept  simple,  such  as  an  "X"  rather  than  picto¬ 
rial  icons  showing  target  type.  Special  symbols  could  be  used  to  represent  ADA 
threats,  and  for  JAAT  operations,  to  show  the  fixed-wing  aircraft  the  target. 

In  addition,  a  digital  terrain  elevation  data  base,  if  available,  could  be  used  to 
continuously  compute  intervisibility  from  known  target  positions  in  the  terrain. 
This  approach  would  be  more  effective  than  waiting  for  the  APR-39  to  indicate  that 
masking  has  not  been  maintained.  One  simple  method  of  portraying  masking  is  to 
depict  the  target  symbols  in  dashed  lines  if  the  aircraft  does  not  have  intervisibility 
and  solid  lines  if  intervisibility  exists.  Such  an  approach  also  serves  to  cue  the  pilot 
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that  the  target  indicated  by  a  conformal  symbol  does  or  does  not  reside  on  the 
nearest  landform  in  the  direction  of  the  target. 

Intelligent  Symbology  Management.  The  appearance  of  target  symbols  of  both 
types  could  be  intelligently  moded,  appearing  either  at  a  given  target  distance,  or  on 
arrival  at  positions  in  the  planned  route  of  flight.  If  there  are  many  targets,  a 
declutter  capability  may  be  necessary,  perhaps  through  a  voice  command  such  as 
"Threats  off."  The  intelligent  masking  symbol  management  would  be  based  on 
continuous  intervisibility  computations. 

New  Spatial/Geographic  Symbols:  Mission  Management  Data 

As  noted  above,  the  pilot  carries  only  a  very  simple  strip  map  of  the  flight 
route,  and  is  dependent  upon  the  CPG  for  most  navigation  functions.  As  a  result,  it 
is  difficult  for  the  pilot  to  maintain  tactical  situation  awareness  in  the  battle  area. 

This  problem  becomes  much  more  critical  when  unplanned  maneuvers  and  route 
changes  are  required,  as  a  result  of  enemy  activity  blocking  a  planned  path  into  or 
out  of  the  BP.  It  is  important  not  to  stray  across  international  boundaries,  division 
or  brigade  boundaries,  phase  lines,  near  enemy  (or  friendly)  ADA  units,  or  into 
planned  artillery  firing  zones.  Most  pilots  agreed  that  it  would  be  extremely 
valuable  to  have  warnings  of  such  dangers  presented  by  conformal  HMD  symbol¬ 
ogy.  The  symbols  should  be  sufficiently  attention-getting  to  be  easily  noticed,  but 
block  as  little  of  the  field  of  view  as  possible. 

Intelligent  Symbology  Management.  The  mission  management  symbols  could 
be  intelligently  moded  to  appear  only  when  the  aircraft  is  at  a  select  distance  from 
the  danger,  (e.g.,  "1  to  3  kilometers")  and  flying  at  a  speed  likely  to  result  in  crossing 
boundaries  within  certain  periods  of  time  (to  be  determined). 

New  Spatial/Geographic  Symbols:  Aircraft  Survival  Equipment  Display 

The  APR-39  radar  warning  device  in  the  AH-64A  provides  voice  warnings  of 
threat  activity  and  their  relative  direction,  such  as  "Searching  -  2  o'clock."  In  order 
to  permit  rapid  masking  of  the  aircraft,  instead  of  receiving  just  a  numerical  bearing 
to  the  threat,  it  "would  be  a  great  idea"  to  use  the  HMD  to  paint  a  symbol  in  the  field 
of  regard,  perhaps  supplemented  with  a  3-D  audio  cue,  to  more  naturalistically 
represent  the  weapon's  position  in  space.  Since  the  APR-39  does  not  provide 
distance-to-weapon  information,  the  symbol  would  have  to  be  a  vertical  line  rather 
than  a  discrete  point  on  the  terrain.  Most  of  those  interviewed  indicated  that  the 
HMD  symbology  could  give  a  spatially  superior  indicant  of  direction  with  a  line  or 
some  other  marker  so  that  the  pilot  can  either  orient  the  aircraft  appropriately  for 
use  of  the  gun,  jammer,  chaff,  or  flares,  or  prepare  to  deploy  to  cover.  Unlike  a 
spoken  warning,  a  visible  line  provides  a  continuous  indicant  of  the  direction  of  the 
threat,  even  as  the  pilot  changes  his  heading  to  respond  to  the  threat. 

Intelligent  Symbology  Management.  The  symbol  should  appear  automatically 
with  the  APR-39  warning,  and  remain  on  "only  for  a  few  seconds"  while  the  pilot 
weighs  his  alternatives  and  chooses  a  course  of  action.  If  the  threat  is  a  preplanned 


2-24 


target,  however,  distance  information  will  be  available  to  position  a  conformal 
target  indicator  symbol. 

Phase  I PNVS  Symbology  Survey 

The  PNVS  symbology  set  used  with  the  IHADSS  has,  for  the  most  part,  been  very 
well  designed  according  the  SMEs  interviewed  at  Fort  Rucker.  That  is  not  to  say  that 
it  is  necessarily  easy  to  learn  and  use.  To  find  out  more  about  the  SME's  impres¬ 
sions  of  specific  symbols,  we  asked  them  to  rate  each  symbol  on  three  seven-point 
scales.  The  SMEs  were  requested  to  check  a  box  scoring  the  utility,  frequency,  and 
design  quality  of  each  of  the  symbols  in  comparison  with  the  entire  symbol  set,  as 
shown  below: 


Of  all  of  the  information  presented  on  the  IHADSS,  this  is  one  of  the 
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The  overall  results  of  this  survey  are  summarized  in  Table  2.2  on  the  following 
page. 

These  ratings  served  to  confirm  the  interview  findings,  demonstrate  the  consistency 
of  IP  impressions,  and  provide  an  objective  basis  for  comparison  of  the  relative 
merits  of  the  various  symbols.  The  utility  and  frequency  ratings  are  expected  to  be 
valuable  in  aiding  decisions  regarding  intelligent  symbology  management 
techniques  for  these  symbols. 

At  the  conclusion  of  the  Phase  I  period  of  performance,  we  had  met  and,  in 
many  respects,  exceeded  our  original  project  objectives.  We  had  (a)  determined  the 
clear  utility  and  feasibility  of  an  innovative,  intelligent  symbology  management 
system,  (b)  developed  a  unique  portable  HMD  symbology  simulator  with  intelligent 
symbology  management  capabilities,  and  (c)  expanded  and  verified  our  findings 
with  intensive  interviews  with  thirteen  expert  AH-64  Apache  pilots.  The  combined 
results  of  our  several  coordinated  research  methodologies  had  led  directly  to  the 
conclusion  that  such  a  system  could  and  should  be  developed,  and  that  it  should  be 
broadly  applicable  to  current  and  future  aircraft. 

In  many  ways.  Phase  II  activities  were  a  direct  outgrowth  of  the  successful 
methods  employed  during  Phase  I.  The  objectives  and  analyses  were  expanded  to 
address  many  more  pilot  functions  and  tasks,  the  lap-top  simulator  was  replaced  by 
a  sophisticated  flight  simulator  with  aircraft  controls  and  an  immersive  HMD 
display,  and  the  successful  interviews  with  SME's  were  expanded  to  include  a  range 
of  experiments  and  demonstrations  to  determine  the  utility  of  new  symbology  and 
the  appropriate  cues  for  intelligently  moding  these  symbols. 
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Table  2.2 

PNVS  Symbols  Ranked  by  Utility,  Frequency  of  Use,  and  Design  Quality 


Rank 

Symbol 

Utility 

1 

Missile  Const  Box 

6.67 

2 

Velocity  Vector 

6.67 

3 

Rocket  Strg  Cursor 

6.58 

4 

LOS  Reticle 

6.58 

5 

Rate  of  Climb  Ind 

6.50 

6 

Horizon  Line 

6.50 

7 

Heading  Scale 

6.50 

8 

Fixed  Lubber  Line 

6.50 

9 

Radar  Alt  (Digital) 

6.42 

1  0 

Weapon  Control 

6.42 

1  1 

Skid/Slip  Ball 

6.25 

1  2 

Range/ Rng  Source 

6.25 

1  3 

Airspeed  (Digital) 

6.17 

1  4 

Acceleration  Cue 

6.17 

1  5 

Vertical  Speed  Scale 

6.17 

1  6 

Engine  Torque 

6.00 

1  7 

Weapon  Status 

6.00 

1  8 

Command  Heading 

5.92 

1  9 

Sight  Status 

5.42 

20 

Alternate  Sensor  Brg 

4.92 

21 

Field  of  View  Box 

4.33 

22 

Field  of  Regard  Box 

4.17 

23 

Cued  LOS  Dot 

4.08 

24 

Radar  Alt  Bugs 

3.92 

25 

Cued  LOS  Reticle 

3.75 

26 

Head  Tracker 

3.33 

27 

Cueing  Dots 

3.33 

28 

Radar  Alt  V.  Scale 

3.25 

29 

Hover  Position  Box 

2.83 

30 

Radar  Alt  V.  Tape 

2.33 

Mean  Rating  5.33 


Symbol 

Frequency 

Velocity  Vector 

6.58 

LOS  Reticle 

6.50 

Acceleration  Cue 

6.50 

Rate  of  Climb  Ind 

6.50 

Missile  Const  Box 

6.42 

Heading  Scale 

6.42 

Fixed  Lubber  Line 

6.42 

Vertical  Speed  Scale 

6.42 

Airspeed  (Digital) 

6.33 

Skid/Slip  Ball 

6.33 

Horizon  Line 

6.25 

Radar  Alt  (Digital) 

6.25 

Rocket  Strg  Cursor 

6.17 

Command  Heading 

6.08 

Weapon  Control 

6.08 

Weapon  Status 

6.00 

Range /Rng  Source 

5.92 

Engine  Torque 

5.50 

Sight  Status 

5.33 

Field  of  View  Box 

4.42 

Alternate  Sensor  Brg 

4.25 

Field  of  Regard  Box 

4.00 

Cued  LOS  Dot 

4.00 

Radar  Alt  Bugs 

3.58 

Head  Tracker 

3.17 

Cued  LOS  Reticle 

3.17 

Radar  Alt  V.  Scale 

3.08 

Cueing  Dots 

2.42 

Hover  Position  Box 

2.00 

Radar  Alt  V.  Tape 

2.00 

5.14 

Symbol 

Design 

Rate  of  Climb  Ind 

6.33 

LOS  Reticle 

6.25 

Airspeed  (Digital) 

6.25 

Skid/Slip  Ball 

6.25 

Missile  Const  Box 

6.17 

Heading  Scale 

6.17 

Fixed  Lubber  Line 

6.17 

Radar  Alt  (Digital) 

6.17 

Velocity  Vector 

6.08 

Acceleration  Cue 

5.92 

Rocket  Strg  Cursor 

5.83 

Vertical  Speed  Scale 

5.83 

Range/Rng  Source 

5.75 

Weapon  Control 

5.67 

Command  Heading 

5.50 

Horizon  Line 

5.50 

Weapon  Status 

5.08 

Field  of  Regard  Box 

5.00 

Field  of  View  Box 

4.92 

Cued  LOS  Dot 

4.75 

Sight  Status 

4.75 

Cueing  Dots 

4.42 

Alternate  Sensor  Brg 

4.33 

Engine  Torque 

4.25 

Head  Tracker 

4.17 

Cued  LOS  Reticle 

4.08 

Hover  Position  Box 

4.08 

Radar  Alt  Bugs 

4.00 

Radar  Alt  V.  Scale 

3.92 

Radar  Alt  V.  Tape 

3.08 

5.22 
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Section  3:  Symbology  Management  Issues 


There  are  two  major  dimensions  of  symbology  management:  the  method  of 
mode  switching  and  the  method  of  information  organization.  The  method  of  mode 
switching  is  that  cue,  somehow  initiated  by  the  pilot  or  some  non-human  portion  of 
the  system  that  causes  symbology  to  change  in  content  or  behavior.  The  method  of 
information  organization  describes  the  manner  in  which  symbology  sets  are 
subdivided  into  groups,  differentially  retrievable  by  the  mode  switching  cue.  These 
two  topics  will  be  described  in  more  detail  in  the  following  pages. 

Comparisons  of  Potential  Pilot-Operated  Mode  Switching  Methods 

Manual  Mode  Controls 

The  number  of  potential  manual  mode-change  input  devices  and  methods  are 
legion.  Nearly  any  type  of  control  could  be  pressed  into  service  for  these  selections, 
including  controls  that  have  existed  for  decades  including  push-buttons,  toggle 
switches,  rotary  selector  switches,  and  thumbwheels.  Because  symbology  mode 
changes  may  be  most  needed  when  manual  workload  is  greatest,  however,  it  is 
reasonable  to  limit  consideration  of  these  methods  to  those  that  can  be  employed 
while  both  hands  are  engaged  with  the  collective  and  cyclic  controls,  otherwise 
known  as  "HOT AS,"  for  "hands  on  throttle  and  stick."  or,  for  rotorcraft,  "HOCAC" 
for  "hands  on  cyclic  and  collective." 

The  placement  of  controls  on  the  cyclic  and  collective,  however,  does  not 
necessarily  ease  the  degree  of  manual  workload  associated  with  their  use.  In  both 
fixed  wing  and  rotary  wing  military  aircraft,  the  two  grip  control  surfaces  are 
crowded  with  diverse  input  devices.  Psychomotor  demands  are  a  serious  human 
factors  problem  in  high-performance  aircraft.  The  demands  are  so  great  that 
HOTAS  procedures  in  general  have  been  likened  by  pilots  to  "playing  the  piccolo" 
in  reference  to  the  rapid  and  complex  finger  movements  that  are  required  in  the  F- 
16C  and  F-15E  (Campbell  and  Rogers,  1993). 

In  the  Apache  helicopter,  manual  switching  might  seem  simple  because  there 
are  only  four  modes  and  nearly  all  of  the  pilot's  typical  symbology  changes  are 
between  two  modes:  Hover  and  Transition.  Nevertheless,  this  change  often  occurs 
during  the  periods  of  heaviest  pilot  workloads  when  thumb  "fumbling"  and  switch 
direction  confusions  are  the  most  prevalent  and  frustrating  (Rogers,  Spiker,  & 
Asbury,  1996).  For  example,  on  landing,  the  Hover  mode  is  used  until  the  main 
gear  touches  the  ground,  when  the  pilot  switches  to  Transition  mode  to  view  the 
horizon  line.  This  is  of  paramount  importance  in  the  slope  landing  during  which 
the  pilot  is  very  busy  controlling  the  roll  rate  and  groping  for  the  manual  mode 
switch  is  an  undesirable  burden.  Furthermore,  if  a  variety  of  new  HMD  symbols  are 
to  be  considered  for  use  in  advanced  rotorcraft,  the  single  thumb  control  would  not 
provide  enough  control  options,  and  it  is  doubtful  that  a  sufficient  number  of 
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additional  controls  could  be  added  to  the  cyclic  or  collective  and  still  be  located 
within  convenient  finger  and  thumb  reach. 

Alternate  Psychomotor  Controls 

In  a  visionary  paper  presented  to  the  Human  Factors  Society  over  a  decade  ago, 
Furness  (1986)  described  a  revolutionary  virtual  crew  station  entitled  the  "Super 
Cockpit."  At  the  time,  Furness  noted  that  although  some  design  criteria  might  be 
derived  or  extrapolated  from  existing  research  findings,  "most  will  require  a  new 
generation  of  human  factors  research  to  acquire  the  empirical  data"  necessary  for 
their  development.  With  reference  to  control  inputs,  the  paper  itemized  five 
possibilities  to  be  used  in  combination  within  a  virtual  cockpit.  These  five  are  still 
the  primary  contenders  for  alternate  psychomotor  controls,  and  are  briefly  described 
below. 

(1)  Head-aimed  control.  In  this  technique,  the  pilot  can  use  the  HMD  to 
position  a  screen-stabilized  reticle  over  a  virtual  (cockpit  referenced)  switch  box  and 
press  an  enabling  switch  to  select  the  function.  We  have  worked  (on  another 
project)  for  several  years  on  this  concept  and  find  it  surprisingly  effective  (Rogers, 
Spiker,  &  Fischer;  1996).  Typical  acquisition  times,  depending  upon  the  size  of  the 
target  and  angular  distance  of  the  head  movement  were  approximately  1  second,  as 
shown  in  Figure  3-1.  This  movement  was  a  full-screen  head  movement  of  10° 
angular  rotation.  Times  are  about  20%  longer  for  such  movements  than  for  5° 
movements,  even  though  more  rapid  head  velocities  are  reached  in  the  longer 
movements.  Target  acquisition  times  for  horizontal  and  vertical  movements  are 
approximately  the  same,  although  diagonal  movements  increase  acquisition  time  by 
about  18%. 

As  might  be  supposed,  target  acquisition  times  are  strongly  dependent  upon 
accuracy  requirements.  Had  our  cursor  been  only  slightly  larger  (about  0.34  inches 
instead  of  0.29  inches  in  diameter,  typical  target  acquisition  times  would  have  been 
reduced  by  about  150  msec.  Any  amount  of  system  lag  between  head  movement 
and  cursor  movement  is  destructive  to  performance,  even  though  the  user  may  be 
unaware  of  the  lag.  We  found  there  to  be  a  linear  relationship  between  lag  and 
target  acquisition  time  such  that  each  millisecond  increment  in  lag  between  head 
movement  and  cursor  response  increased  acquisition  time  by  about  7  milliseconds 

Once  the  cursor  has  reached  the  target,  the  stability  of  the  head-controlled 
cursor  has  been  shown  to  be  reasonably  good.  For  example.  Figure  3-2  shows  a  ten- 
second  time  history  of  the  cursor  stability  without  external  perturbations  such  as 
vehicle  movement  or  vibration.  Such  findings  suggest  that  using  head  dwell  time 
to  actuate  a  control  (as  opposed  to  an  additional  manual  switch)  might  prove 
feasible. 
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Figure  3-1.  Head  movement  velocity  profile  for  a  10-degree  head  movement  to 
place  a  cursor  on  a  small  target  (Rogers,  Spiker,  &  Fischer;  1996). 


Figure  3-2.  A  10-second  time  history  for  one  subject  aiming  at  a  stationary  target 

with  a  head-mounted  sight.  The  circle  represents  0.57°  or  34  arc-minutes 
(Wells  &  Griffin,  1987). 
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In  our  studies,  the  head-tracked  cursor  technique  was  shown  to  be  considerably 
faster  than  the  thumb-operated  cursor  control  currently  in  use  in  the  F-16C  and  F- 
15E.  However,  the  extent  to  which  buffeting  and  high-g  maneuvering  (in  the 
fighter  aircraft)  or  vibration  (in  the  helicopter)  would  effect  accuracy  or  even 
usability  of  this  approach  has  not  been  fully  evaluated.  Another  potential 
disadvantage  is  the  clutter  introduced  by  an  array  of  virtual  icons  used  for  selection 
of  symbol  display  options.  Such  an  array  could  be  positioned  in  a  head-down 
position  within  the  cockpit  to  reduce  clutter  of  the  outside  terrain,  but  would  then 
lose  the  primary  virtue  of  the  HMD:  head  up  and  out  of  the  cockpit. 

(2)  Eye  control  system.  Using  a  sensor  to  determine  the  instantaneous  position 
of  the  pilot's  eyes  combined  with  head  position  data  would  identify  eye  fixation 
angles  within  the  cockpit  so  that  the  pilot  could  look  at  either  virtual  or  real 
switches  and  manipulate  them.  With  this  approach,  an  icon  would  appear  in  a 
known  position  on  the  HMD  field  of  view.  Using  the  independent  eye  tracker,  the 
pilot  can  actuate  this  "switch"  to  change  symbology  modes  by  maintaining  his  gaze 
on  the  icon  for  a  set  period  of  time.  Variables  to  be  considered  with  this  option 
include  the  screen  location  of  the  switches,  their  size  and  shape,  as  well  as  the  gaze 
time  required  for  actuation.  Alternatively,  the  pilot  could  actuate  them  with  a 
verbal  command  such  as  "on,"  "off,"  or  "select,"  or  use  a  dedicated  manual  switch  to 
activate  or  deactivate  the  switch  currently  being  viewed. 

Calhoun  &  Janson  (1986),  for  example,  performed  an  experiment  in  which  six 
subjects  selected  various  discrete  switches  on  the  front  panel  of  a  cockpit  simulator 
during  a  manual  target  tracking  task.  Their  subjects  were  instructed  to  direct  their 
gaze  at  the  switch  designated  by  an  auditory  cue  and  then  to  make  a  "consent  input" 
(in  the  form  of  either  a  manual  response  or  a  verbal  utterance).  In  the  control  con¬ 
dition,  simulating  a  standard  manual  control,  subjects  simply  actuated  the  switches 
with  their  left  hand.  The  authors'  analysis  of  mean  switching  time  showed  no  sig¬ 
nificant  differences  among  the  three  switching  methods.  Since  the  eye  control 
method  was  not  slower  than  the  manual  method,  the  authors  concluded  that  an 
eye-tracked  control  system  would  be  a  feasible  option  when  hands-off  control  is 
needed.  The  authors  suggested  further  study  of  the  advisability  of  having  multiple 
control  mechanisms  available,  so  that  the  pilot  could  choose  which  method  to 
employ. 

The  eye  control  system  is  similar  in  concept  to  the  head-aimed  control,  but  uses 
existing  and  very  well-trained  skills  whereas  head  tracking  is  somewhat  less  natural 
and  some  practice  is  required  to  master  the  skills.  Furthermore,  eye  movements  are 
considerably  faster  than  head  movements,  so  the  selection  of  the  functions  should 
theoretically  proceed  more  rapidly,  although  this  was  not  the  case  in  the  Calhoun  et 
al.  study.  It  is  important  to  remember,  however,  that  the  technologies  used  for  head 
tracking  have  proved  over  the  years  to  be  robust  and  survivable  in  the  cockpit 
environment,  while  the  same  claim  can  not  yet  be  made  for  the  eye-tracking 
technologies. 

(3)  Voice-actuated  control.  The  pilot  can  utter  specific  control  commands  to  a 
system  monitoring  his  speech  when  a  separate  switch,  virtual  or  physical,  is 
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enabled.  Although  a  great  deal  has  been  written  about  voice  technology  in  recent 
years,  the  classic  analysis  of  the  utility  of  voice  technology  in  Army  helicopters 
remains  that  of  Dr.  Alan  Spiker  (1986)  of  Anacapa  Sciences.  His  report  describes  the 
results  of  a  series  of  human  factors  analyses  designed  to  evaluate  the  feasibility  of 
applying  voice  technology  to  Army  helicopter  cockpits.  The  primary  purpose  of  this 
work  was  to  specify  the  functional  requirements  to  be  satisfied  by  such  applications, 
and  to  identify  those  tasks  for  which  aviator  workload  would,  and  would  not  be, 
reduced  by  this  technology.  The  end  result  of  this  eight-month  effort  was  a  set  of 
recommendations  and  guidelines  that  delineated  the  Army  aviation  task  areas  in 
which  application  of  speech  recognition  and  speech  synthesis  technology  is  likely  to 
have  either  high  payoff,  low  payoff,  or  no  payoff. 

It  is  important  to  note  that  in  conducting  this  study  it  was  assumed  that  the 
technology  itself  was  mature  and  functioned  rapidly  and  without  error.  Thus  the 
results  are  not  limited  to  the  technology  of  the  1980s,  but  are  still  quite  valid.  In 
short,  it  was  found  that  the  major  obstacles  to  the  application  of  voice  systems  are 
task-  and  user-based  rather  than  technological.  The  primary  factors  ultimately 
limiting  voice  system  applications  are  human  factors  rather  than  technological  ones. 

First,  voice  technology  is  most  applicable  to  tasks  that  impose  high 
sensory/ motor  workload  and  least  applicable  to  tasks  requiring  high  cognitive 
workload.  Voice  systems  can  only  provide  input  to  a  control  device  or  report  the 
output  from  an  external  sensor;  hence,  they  require  an  existing  level  of  automation 
to  be  applicable.  Since  these  systems  do  not  automate  decision-making  for  the 
aviator,  their  use  will  necessarily  be  restricted  to  tasks  in  which  the  primary 
workload  is  sensory  or  motor  rather  than  cognitive. 

Second,  due  to  recent  avionics  enhancements,  much  of  the  aviator's  workload 
is  now  cognitive  rather  than  sensory  or  psychomotor.  As  avionics  enhancements 
have  addressed  many  of  the  sensory  and  psychomotor  difficulties  found  in  previous 
cockpits,  the  cognitive  demands  imposed  by  the  more  sophisticated  equipment  are 
now  the  primary  bottleneck  to  improving  the  workload  situation  in  the  cockpit. 
Some  of  the  most  demanding  cognitive  tasks  include  target  tracking,  monitoring 
approach  areas,  confirming  navigation  data,  aligning  sensor  sight,  selecting  sensor 
search  area,  verifying  launch  constraints,  and  monitoring  laser  designation  codes. 
Unfortunately,  these  tasks  are  also  the  most  resistant  to  automation  through  voice 
since  data  integration  rather  than  I/O  reduction  is  needed  most. 

The  project  methodology  included  analysis  of  the  overload  frequency  and 
duration  data  from  the  60  high-workload  tasks  in  the  LHX  data  base  to  quantify  the 
extent  to  which  voice  automation  would  reduce  aviator  overload.  The  amenability 
of  each  task  to  workload  reduction  via  voice  automation  was  determined  by  jointly 
considering  (1)  application-relevant  findings  from  research  literature,  (2)  first-hand 
discussions  with  aviators,  and  (3)  human  factors  application  principles.  Regarding 
the  latter,  voice  technology  was  considered  applicable  to  a  particular  task  when: 

•  the  combined  visual /motor  workload  is  high 

•  the  psychomotor  movement  is  awkward 
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•  visual  scanning  is  difficult 

•  the  task  involves  extended  psychomotor  activity 

•  a  complex  auditory  discrimination  is  required 

•  a  complex  visual  discrimination  is  required. 

On  the  other  hand,  negative  decisions  concerning  the  application  of  voice 
technology  were  based  in  part  on  the  following  criteria: 

•  unaided  visual  scanning  is  required 

•  the  task  is  primarily  cognitive,  decision-oriented 

•  periods  of  sustained  voice  communication  are  required 

•  rapid  fine  motor  movements  are  needed 

•  psychomotor  adjustments  of  nonautomated  systems  are  required 

•  the  task  involves  recall  of  information  from  memory. 

Despite  the  overall  poor  prognosis  for  cockpit  applications  of  voice  systems, 
four  task  areas  were  identified  for  which  voice  technology  should  prove  quite 
useful:  display  of  threat  identification  and  location  information;  display  of  aircraft 
heading,  altitude,  and  fuel  under  mission-specific  conditions;  automatic  recording 
of  all  radio  voice  messages;  and  spoken  input  as  an  alternative  to  keyboard  entry  of 
digital  target  handoff  messages. 

Human  factors  analyses  revealed  that  a  wide  array  of  task  areas  would  benefit 
little  from  voice  applications,  including:  selection  and  control  of  weapons  and  radio; 
operation  of  sensors;  speech  display  of  checklist  information;  entry  of  target  and 
navigation  data;  display  of  warnings,  cautions,  and  advisories;  and  selection  of 
alternative  message  formats.  In  addition,  five  task  areas  would  be  poorly  served  by 
voice  automation:  target  tracking,  weapon  release,  monitoring  of  terrain  and  non¬ 
automated  cockpit  equipment,  flight  control,  and  display  of  status  information. 

The  utility  of  application  of  voice-actuated  control  for  the  selection  of  symbols 
or  symbology  modes  is  not  clear-cut.  HMD  symbology  may  be  called  for  to  support  a 
range  of  different  tasks,  some  of  which  meet  the  criteria  for  voice  technology 
application,  and  some  that  do  not.  The  pilot's  manual  output  burden  is  certainly 
reduced,  but  his  cognitive  activity  may  be  interrupted.  Since  such  a  system  does  not 
automate  decision-making  for  the  aviator,  its  use  provides  the  most  benefit  for  tasks 
in  which  the  primary  workload  is  sensory  or  motor  rather  than  cognitive. 

Since  sensory  and  motor  tasks  in  the  cockpit  are  becoming  overshadowed  by 
cognitive  tasks,  at  least  in  terms  of  pilot  workload,  the  ideal  option  would  seem  to 
be  an  automatic,  intelligent  system  to  automate  decision-making  and  present  and 
remove  symbology  in  the  most  effective  manner.  Nevertheless,  most  of  the  pilots 
we  have  interviewed  over  the  course  of  this  project  would  demand  either  a  manual 
or  a  voice  control  mechanism  to  override  an  automated  symbology  presentation 
system  when  necessary.  Given  the  shortage  of  space  for  supplementary  manual 
controls,  the  voice  command  option  remains  a  viable  one. 
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(4)  Touch-sensitive  panel.  Furness  (1986)  suggested  the  use  of  a  touch-sensitive 
panel  such  that  the  pilot  places  a  finger  on  a  physical  surface  area  "with 
superimposed  cockpit-stabilized  visual  information"  from  the  HMD  and  uses  these 
"virtual  buttons"  to  call  up  information  or  other  cockpit  switching  functions.  In 
accordance  with  the  pilot's  finger  pressure  on  various  alternatives,  "touch  panel 
regions  are  redefined  for  different  additional  panels  which  may  be  called  up  and 
windowed  within  the  display."  In  effect,  a  multi-function  display  can  be  created 
with  virtual  labels  instead  of  self-illuminated  CRT  or  flat  panel  displays  with 
adjacent  buttons  of  pressure-sensitive  overlays.  Although  the  authors  have  not 
found  any  experimental  comparisons  of  virtual  labels  on  touch  panels  with  other 
control  systems,  self-illuminated  touch  screens  have  been  developed  and  studied. 

Liggett,  Kustra,  Reising,  &  Hartsock  (1997)  compared  the  use  of  a  thumb- 
controlled  cursor,  a  head-tracked  cursor,  and  a  touch-sensitive  screen  for  the 
designation  of  targets  on  a  10  inch  by  10  inch  CRT  screen  and  found  that  the 
designation  via  the  touch  screen  was  significantly  faster  than  either  of  the  other  two 
devices.  The  nature  of  the  task,  however,  was  not  ideal  for  evaluating  rapid 
response  times  of  skilled  motor  outputs,  since  the  complexity  of  the  task  typically 
required  over  15  seconds  to  perform  in  full  and  had  a  number  of  cognitive 
elements.  Furthermore,  because  the  one-size  helmet  fit  some  subjects  poorly,  a  fair 
evaluation  for  the  head-tracker  mechanism  was  not  permitted.  Nevertheless,  it  is 
evident  that  a  touch-sensitive  screen  can  be  an  effective  input  device  if  HOTAS  or 
HOCAC  is  not  required. 

The  touch-sensitive  menu  concept  has  been  used  to  good  advantage  in  the 
RAH-66  Comanche  design  (Friedman,  Hamilton,  Rice,  and  Stockdale,  1992). 
Although  the  control  labels  are  self-illuminated  rather  than  virtual,  the  ten 
software  controlled  switches  known  as  the  Touchscreen  Menu  Items  (TMI)  are 
located  for  easy  access  in  the  lower  portion  of  the  leftmost  multi-purpose  display. 

Each  switch  can  provide  a  variety  of  types  of  functions  and  is  graphically  coded 
to  show  whether  it  is  an  "action,"  "toggle,"  "menu  prompt,"  "state  select,"  or 
"keyboard  prompt"  switch.  The  codes,  labels,  and  functions  change  to  reflect  the 
available  options  for  the  operations  underway,  thus  providing  an  endless  number 
of  possible  inputs  within  a  very  small  panel  space.  For  example,  Figure  3-3  shows 
the  TMI  configured  for  use  with  the  "PIREP"  window  of  the  multi-purpose  display. 
The  asterisks  on  the  TMI  indicate  "action"  keys  that  command  the  system  to 
perform  a  one-time  task  such  as  "send-urgent." 

The  TMI,  while  too  complex  to  be  described  here  in  detail,  is  a  good  example  of 
striving  to  meet  the  challenge  of  maintaining  pilot  awareness  of  the  automatically 
changing  system  status.  This  is  especially  vital  given  the  wide  range  of  control  and 
display  interface  possibilities  and  the  limited  amount  of  cockpit  space.  While  the 
TMI  does  not  employ  virtual  labels,  the  content  and  format  of  the  control  labeling, 
the  compatibility  of  its  structure  with  the  task  at  hand,  and  its  effective  use  of 
human-computer  dialog  principles  could  be  expected  to  be  much  more  important 
than  the  source  of  the  illumination  itself. 
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Figure  3-3.  The  Comanche  TMI  adjacent  to  the  multi-purpose  display. 

(5)  Virtual  hand  controller.  Using  a  special  glove  and  a  magnetic  tracker,  hand 
position  and  orientation  can  be  sensed  in  six  degrees  of  freedom.  Furness  (1986) 
suggested  that  "When  the  hand  is  put  into  a  predetermined  volume  or  region 
within  the  cockpit,  a  three-dimensional  virtual  control  panel  is  windowed  into  the 
visual  display.  The  pilot  then  activates  functions  or  makes  verinier  adjustments  by 
moving  his  hand  or  placing  a  finger  over  the  virtual  switch."  Wearing  such  a 
device,  the  pilot  could  activate  or  adjust  virtual  switches  with  auditory,  visual,  or 
even  tactile  feedback  of  his  actuations. 

Most  applications  would  first  require  measurement  of  the  position  and 
orientation  of  the  forearm  in  space.  To  meet  this  requirement,  some  type  of  optical, 
magnetic,  or  ultrasonic  tracking  sensors  are  positioned  on  the  glove  wristband  or 
back.  In  one  study,  (Liggett,  Reising,  Beam,  &  Hartsock,  1993)  compared  an 
ultrasonic  hand  tracker  with  a  joystick  and  found  that  the  hand  tracker  provided  the 
best  performance  with  respect  to  total  target  designation  time,  (although  the  mean 
time  difference  between  the  two  devices  was  small).  The  nature  of  the  task, 
identifying  a  target  in  a  three-dimensional  space  was  more  compatible  with  a  hand 
movement  in  space  than  the  required  inputs  to  a  3-axis  joystick  device. 

The  technology  continues  to  improve  and  in  addition  to  simple  location  of  the 
hand,  flexible  sensors  are  used  to  accurately  and  repeatably  measure  the  position  and 
movement  of  the  fingers  and  wrist.  The  sensors  are  extremely  thin  and  flexible  and 
produce  almost  undetectable  resistance  to  bending.  The  more  advanced  models 
feature  bend  sensors  on  each  finger,  abduction  sensors,  sensors  measuring  thumb 
crossover,  palm  arch,  wrist  flexion,  wrist  abduction,  and  the  flexion  of  the  distal 
joints  on  the  four  fingers.  An  example  of  one  such  device  is  shown  in  Figure  3-4. 
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The  authors  were  unable  to  obtain  any  experimental  studies  employing  such  a 
device  in  a  cockpit  setting. 


Figure  3-4.  The  “CyberGlove”  (Virtual  Technologies). 

Like  the  touch-sensitive  screen,  it  is  evident  that  a  virtual  hand  controller  can 
be  an  effective  input  device  if  HOTAS  or  HOCAC  is  not  required. 

Summary.  The  existing  control  approach,  and  five  new  control  options 
described  above  have  been  considered.  A  brief  summary  of  their  advantages, 
disadvantages,  and  associated  workloads  are  presented  in  Table  3.1.  From  this 
summary,  it  can  be  seen  that  none  of  these  control  technologies  is  ideal,  and  each 
has  its  limitations.  Use  of  the  touch  panel  and  virtual  hand  controller  require  that 
one  hand  be  moved  from  the  flight  controls.  Head-tracked  and  eye-tracked  control 
may  suffer  from  aircraft  vibration.  Voice  recognition  control  requires  that  the  pilot 
memorize  the  set  of  available  commands,  and  may  interfere  with  voice 
communications.  There  is  very  limited  space  for  additional  manual  (HOCAC) 
controls,  and  manual  workload  is  already  very  high. 

The  key  issue  at  hand  is  how  well  these  control  technologies  might  perform 
when  employed  for  the  selection  of  symbols  or  symbology  sets  immediately  needed 
by  the  pilot.  In  one  regard,  the  requirement  is  simple:  the  symbols  typically  need 
only  to  be  turned  on  or  off,  and  such  discrete  inputs  are  more  easily  dealt  with  than 
complex  motor  tasks  such  as  target  tracking.  When  only  four  symbol  modes  are 
available,  as  in  the  IHADSS  symbology,  a  manual  control  is  adequate,  although  not 
ideal  given  the  current  manual  workload  requirements. 

However,  if  consideration  is  given  to  controlling  symbols  individually,  there  is 
a  large  number  of  symbols  potentially  in  need  of  such  control,  and  the  manual 
control  method  (whether  real  or  virtual)  is  unlikely  to  be  feasible.  In  this  case,  either 
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the  options  must  be  visually  presented  to  the  pilot  for  his  selection,  or  he  must 
recall  the  necessary  command  from  memory.  Presentation  of  a  symbol  list  (whether 
by  text  or  icon)  leads  to  visual  clutter  of  the  HMD  field  of  view  if  the  list  is  presented 
head-up,  and  a  loss  of  view  of  the  outside  world  if  presented  head-down.  The  head- 
tracked,  eye-tracked,  and  touch-panel  technologies  are  all  subject  to  these  difficulties. 
The  remaining  technology,  voice  recognition,  can  provide  no  list,  so  that  its  success 
is  dependent  upon  the  pilot's  ability  to  access  the  desired  symbol  commands  from 
memory  while  performing  other  tasks  with  competing  cognitive  requirements. 


Table  3.1 

A  Summary  Comparison  of  Control  Technologies 


Control  Technology 

Advantages 

Disadvantages 

Workload 

Manual  (HOCAC, 
HOTAS) 

Readily  available,  no 
visual  requirement 

High  manual  workload, 
conflicts,  limited  space 

Cognitive,  Motor 

Head-Tracked 

Fast,  possibly  hands-free 

Vibration  problems, 
potential  clutter 

Cognitive, 

Motor,  Visual 

Eye-Tracked 

Fast,  possibly  hands-free 

Vibration  problems, 
potential  clutter 

Cognitive, 

Visual 

Voice  Recognition 

Versatile,  possibly  hands¬ 
free 

Memory  recall  for 
commands,  potential 
task  interference 

Cognitive,  Motor 

Touch  Panel 

Unlimited  content,  useful 
guidance 

One  hand  off  a/c 
controls,  head  down 

Cognitive, 

Motor,  Visual 

Virtual  Hand  Controller 

Unlimited  content,  control 
types 

One  hand  off  a/c 
controls,  special  glove 

Cognitive, 

Motor,  Visual 

The  final  column  of  Table  3.1  is  instructive.  Each  of  the  control  technologies 
depends  upon  the  pilot's  expenditure  of  two  or  three  types  of  workload  resources; 
cognitive,  motor,  and  visual.  While  it  is  evident  that  adding  to  the  pilot's  workload 
in  order  to  select  a  symbol  that  may  reduce  his  workload  is  less  than  ideal,  the 
specific  impacts  of  these  increments  in  workload  is  difficult  to  determine. 

We  have  found  that  the  most  illuminating  method  of  evaluating  the  specific 
impacts  of  control  technology  use  on  mission  task  performance  is  through 
examination  of  a  Task  Analysis  and  WorkLoad  (TAWL)  data  base  (e.g.,  Rogers, 
Spiker,  Bierbaum,  &  Dunleavy,  (1996)  which  identifies  mission  segments,  functions, 
and  tasks  along  a  timeline.  For  each  task,  the  cognitive,  psychomotor,  visual,  and 
auditory  workloads  are  defined.  As  segments  may  involve  the  concurrent 
performance  of  several  tasks,  the  workload  ratings  for  all  concurrent  tasks  are 
summed.  Figure  3-5  presents  a  small  sample  from  hundreds  of  pages  of  Mission 
Timeline  Reports,  showing  the  summation  of  workload  components  over  three 
concurrent  tasks.  In  this  particular  example,  the  total  cognitive  workload  is  7.7, 
considered  to  be  quite  high,  and  the  inclusion  of  any  task  with  yet  additional 
cognitive  burdens  should  be  avoided. 
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Mission  Timeline  Report 

Segment  12:  Approach  (Contour) 

Time  Operator  Activity  Workload 

1 

30:33:00  j  Pilot 

Iftneitx**  Perlonp  Hov*r{i6| 

|  t«*  — +  Main^rr  Cbsiad#  CJoaranc©  |32| 

[  ^ ww  PerkxTn  External  C&Tmjrvcalion  (79 \ 

TransTifc  Rapol  (191) 

S  mo  nil  of  External  V*&ial  Scene  (230) 

j  Mannar  Vtsual  Scar*  {5551 

1  WtKVfa*?  TOUU 

coc  vi*  aos  oil 

1.2  2.4  i  .<S  C . C  i.l 

5.3  2.2  0.0  43  11,1 

1.2  D  ...  0  1  .  D  D  O  2,2 

7.7  2.0  i.y  It.* 

i _ : _ : _ : _ i 

Figure  3-5.  Sample  section  from  a  TAWL  mission  timeline  report. 

While  these  kinds  of  precise,  in-depth  analyses  are  of  great  value  in  identifying 
the  best  options  is  specific  circumstances,  in  attempting  to  select  a  single,  most 
appropriate,  general  type  of  technology  for  symbology  control  it  must  be  recognized 
that  (a)  the  types  of  workload  most  likely  to  be  experienced  during  use  of  the  control 
are  not  entirely  predictable  and  (b)  the  moment  in  the  task  sequence  when  new 
symbols  will  be  selected  is  not  known  with  certainty. 

Thus,  if  pilot-initiated  actions  are  necessary  for  symbology  selection  it  would 
seem  that  rather  than  attempting  to  predict  the  best  general-purpose  technique, 
multiple  options  should  be  provided  when  possible.  In  this  manner,  at  least  the 
motor  and  visual  workloads  might  be  reduced  by  appropriate  use  of  selection 
techniques.  However,  it  may  not  be  feasible  to  offer  the  pilot  the  option  of  multiple 
symbology  control  methods. 

Finally,  the  cognitive  workload  imposed  by  pilot-initiated  symbology  changes 
cannot  be  avoided  through  use  of  any  of  the  control  technologies  described  above. 
Every  one  of  these  technologies  take  a  toll  on  the  pilot's  cognitive  resources. 

Whether  he  is  using  his  hand,  head,  eye,  or  voice,  he  must  still  recognize  the  need 
for  a  symbology  change,  select  the  necessary  information  element  from  memory, 
identify  the  control  action  required  and  issue  this  action. 

Intelligent  Symbology  Moding 

One  of  the  central  themes  of  this  project  has  been  to  examine  the  potential  of 
an  "intelligent  agent"  to  control  the  flow  of  information  via  the  presentation  or 
removal  of  symbols  from  the  HMD  based  upon  the  occurrence  of  sensed  events 
during  a  mission.  The  importance  of  such  an  approach  should  not  be 
underestimated.  Its  achievement  eliminates  every  one  of  the  disadvantages  and 
workload  components  itemized  in  Table  3.1.  All  of  the  potential  problems  with 
taking  one  hand  off  the  controls,  special  gloves,  new  buttons  in  limited  space,  need 
for  recall  of  commands,  additional  head-down  time,  visual  clutter  from  symbol 
selection  options,  and  vibration  effects  on  symbol  selection  are  simply  swept  away  by 
intelligent  moding.  Even  more  important,  unlike  all  of  the  pilot  initiated  methods. 
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intelligent  moding  requires  no  psychomotor,  visual,  auditory,  or  cognitive 
workload. 

To  date,  two  extremes  of  the  intelligent  aid  spectrum  have  been  represented  by 
the  Comanche,  and  the  Rotorcraft  Pilot's  Associate  (RPA)  programs.  The  Comanche 
HIDSS  (Helmet  Integrated  Display  Sight  Subsystem)  symbology  presents  all 
information  necessary  for  flight,  navigation,  target  acquisition,  fire  control,  and 
degraded  modes.  The  basic  flight  modes,  NOE,  Cruise,  Approach,  and  IMC,  are 
selected  by  pressing  switches  on  a  "touch  menu"  on  the  aircraft  console.  Within  the 
NOE  and  Cruise  modes,  declutter  options  are  available.  The  Comanche  system  has 
been  provided  with  some  initial  symbology  automation  features.  For  example,  the 
radar  altimeter  (digital  and  analog)  displays  are  present  in  the  NOE  and  Cruise 
modes  when  the  aircraft  absolute  altitude  is  500  feet  or  less  AGL,  and  disappear 
above  500  feet.  This  is  actually  similar  in  concept  to  the  disappearance  of  the  Apache 
vertical  tape  and  scale  above  200  feet. 

In  addition,  when  Auto  is  selected  in  the  Comanche,  the  displayed  symbology 
set  depends  upon  ground  speed.  At  ground  speeds  greater  than  40  knots,  the  cruise 
symbology  set  is  automatically  presented.  At  ground  speeds  less  than  40  knots,  the 
symbology  set  automatically  switches  to  NOE.  The  flight  path  vector  symbol  is 
presented  at  ground  speeds  greater  than  10  knots  (except  in  NOE  declutter).  Also,  if 
a  crew  member  selects  a  weapon,  the  appropriate  weapon  symbology  is  added  to  the 
flight  symbology.  Thus,  detection  of  the  simple  events  such  as  altitude  and  airspeed 
changes  and  pilot  weapon  selections  can  be  used  to  aid  in  symbology  management. 

In  contrast  to  these  simple  rules,  the  RPA  program  has  aimed  to  use  intelligent 
aids  to  achieve  revolutionary  improvements  in  combat  helicopter  mission 
effectiveness  by  significantly  reducing  the  cognitive  workload  for  the  crew.  This 
reduction,  in  turn,  requires  that  a  knowledge-based  Cognitive  Decision  Aiding 
System  (CDAS)  be  integrated  with  many  of  the  helicopter's  important  subsystems. 
The  CDAS  would  reconfigure  HMD  symbology  in  response  to  external  and  internal 
events  undetected  by  the  pilot. 

The  CDAS  element  that  would  automatically  control  in-flight  changes  to  HMD 
symbology  is  the  Cockpit  Information  Manager  (CIM).  The  CIM  has  been  designed 
to  monitor  mission  context  and  pilot  actions  that,  in  turn,  support  the  crew  with 
timely  information  and  automated  aids.  Within  the  CIM  itself,  a  Display  Manager 
would  prioritize  the  information  needs  for  each  crew  member's  tasks  and  select  the 
display  formats  that  satisfy  the  most  important  information  requirements  with  the 
smallest  increase  in  task  load.  As  a  consequence,  the  HMD  will  have  many  more 
modes  and  sub-modes  than  a  traditional  HMD.  From  a  human  factors  standpoint, 
the  challenge  will  be  to  ensure  that  any  changes  in  symbology  appearance,  location, 
and  format  constitute  a  natural  aid  to  the  pilot's  tasks  and  do  not  disrupt  or 
disorient  him  in  any  way. 

Between  the  halting  first  steps  in  intelligent  information  presentation 
represented  by  the  Comanche,  and  the  lofty,  but  distant,  goals  sought  by  the  RPA, 
there  lies  a  great  range  of  near-term  opportunity  for  improving  symbology 
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management  through  use  of  intelligent  systems.  With  the  integration  of  aircraft 
systems  through  the  1553  bus,  it  is  possible  to  sense  hundreds  of  key  events  that 
could  be  used  to  identify  the  state  and  actions  of  the  aircraft  and  its  controls,  and  to 
infer  the  "pilot's  intent"  regarding  activities  to  be  performed  in  the  immediate 
future.  These  events  and  inferences  could  form  a  set  of  "leading  indicators"  used  to 
predict  the  most  useful  sets  of  symbols  on  the  HMD.  The  initial  development  and 
evaluation  of  such  an  innovative  and  intelligent  information  presentation  system 
for  an  HMD  has  been  the  central  objective  of  this  project. 

Thus,  instead  of  revising  the  subsets  of  symbols  within  various  pre-set  modes, 
or  forcing  the  pilot  to  deal  with  an  ever  greater  set  of  modes  and  symbology 
management  tasks,  an  essentially  "modeless"  symbology  management  system  could 
be  developed,  using  an  intelligent  system  to  base  symbology  presentation  upon  the 
pilots  immediate  information  requirements. 

Symbology  Information  Organization  (Moding)  Strategies 

The  organization  of  information  elements  into  sets  or  "modes"  that  are 
individually  retrievable  for  the  pilot's  use  is  intended  to  provide  groups  of  symbols 
that  are  suited  for  various  phases  or  segments  of  the  mission.  This  approach  has 
resulted  in  various  levels  of  success  depending  upon  the  quality  of  the  information 
analysis  employed  and  upon  the  suitability  of  the  tasks  for  such  division  of 
information  elements.  There  is  clearly  a  trade-off  between  a  display  over-crowded 
with  symbols  and  the  workload  of  recalling  the  various  modes  and  switching  to 
them  at  the  appropriate  time. 

In  the  case  of  rotorcraft  HMDs,  the  moding  solutions  employed  have  resulted 
in  very  few  changes  of  symbols  between  modes.  Such  a  situation  may  well  merit 
the  moding  of  individual  symbols  by  intelligent  recognition  of  the  current  utility  of 
an  information  element.  These  concepts  are  further  explored  in  the  following 
pages. 

Comparing  Moding  of  the  Apache,  Comanche,  and  ANVIS/HUD 

As  a  background  to  the  analysis  of  symbology  information  organization,  it  is 
illuminating  to  compare  the  three  existing  Army  HMD  symbology  moding.  The 
Apache,  Comanche,  and  ANVIS/HUD  each  use  quite  different  techniques  for 
selection  of  HMD  symbols.  These  three  moding  approaches  are  compared  in  the 
figures  and  tables  that  follow. 

PNVS  symbology  modes,  cues,  and  symbols.  Many  of  the  PNVS  symbology 
elements  are  shown  in  all  modes,  including  the:  heading  scale,  lubber  line,  alternate 
sensor  bearing,  command  heading,  rate  of  climb  indicator,  radar  altitude  digital  and 
analog  indicators,  skid/ slip  ball  and  lubber  lines,  sensor  field  of  regard  box,  field  of 
view  box,  cued  LOS  reticle  and  dots,  airspeed  readout,  engine  torque,  and  head 
tracker  diamond.  Additional  cues  are  provided  in  each  of  the  four  moding 
operations,  shown  in  Figure  3-6a  through  3-6d. 
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Hover  Mode 


Added  Symbology  in  Hover  Mode:  6-Knot  Velocity  Vector  and  Acceleration  Cue 


o 


Optional  Symbology  in  Hover  Mode 
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■  1  1 

I _ 1 

•  • 

□□□□□□mnonn  _ 

□□□□□□□□□□□a 

• 

Unused  Symbology  in  Hover  Mode  —  _ 
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Figure  3-6a.  The  Hover  Mode  screen. 
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Transition  Mode 
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Added  Symbology  in  Transition  Mode: 

60-Knot  Velocity  Vector,  Acceeration  Cue,  and  Horizon  Line 


Optional  Symbology  in  Transition  Mode 


l - 1 


□□□□□□□□□□□a 


□□□□□□□□□□□a 


Unused  Symbology  in  Transition  Mode 


Figure  3-6b.  The  Transition  Mode  screen. 
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Cruise  Mode 


Added  Symbology  in  Cruise  Mode:  Horizon  Line 


Optional  Symbology  in  Cruise  Mode 
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Unused  Symbology  in  Cruise  Mode  O 

□ 

Figure  3-6c.  The  Cruise  Mode  screen. 
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Bob-Up  Mode 


Added  Symbology  in  Bob-Up  Mode:  O 

6-Knot  Velocity  Vector,  Acceleration  Cue,  and  Hover  Position  Box 


Optional  Symbology  in  Bob-Up  Mode 


■ 


□□□□□□□□□□□□  □□□□□□□□□□□□ 


Unused  Symbology  in  Bob-Up  Mode 


Figure  3-6d.  The  Bob-Up  Mode  screen. 
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The  apparent  complexity  of  the  Apache  symbology  leads  less  experienced 
observers  to  assume  that  substantial  changes  are  made  to  the  symbol  sets  when 
moving  from  one  mode  to  another.  Listing  the  symbols  present  in  each  mode  tends 
to  exaggerate  their  differences  rather  than  their  similarities.  Table  3.2  clarifies  the 
nature  of  the  Apache  moding  structure.  Rather  than  depicting  all  of  the  symbols 
that  are  presented  in  each  mode,  the  symbols  that  are  not  presented  are  identified  by 
the  letter  "N". 

This  table  shows  that  the  Apache  moding  actually  changes  the  presentation  of 
only  four  of  the  approximately  28  symbols:  the  velocity  vector,  acceleration  cue, 
hover  position  box,  and  horizon  line.  The  most  frequently  performed  mode  switch, 
between  Hover  and  Transition  changes  only  the  presence  of  the  horizon  line, 
velocity  vector,  and  acceleration  cue. 

The  enablement  of  the  other  eight  symbols  that  may  potentially  be  presented 
depends  not  on  moding  selection  with  the  thumb  control,  but  on  control  selections 
made  during  weapons  use.  A  case  could  be  made  that  these  eight  symbols  are 
already  intelligently  managed  because  they  appear  only  when  the  pilot's  intent  is 
deduced  by  the  system. 

Table  3.2 

Apache  Symbology  Changes  from  Mode  Switching 


AH-64  PNVS  Symbology 

Modes 

Cruise 

Transition 

Hover 

Bob-up 

1.  Position  and  Movement 

Line  of  Sight  (LOS)  Reticle 

Airspeed  Digital  Readout 

Velocity  Vector 

N 

60  kts 

6  kts 

6  kts 

Acceleration  Cue 

N 

Hover  Position  Box 

N 

N 

N 

2.  Attitude/Altitude 

Engine  Torque  Digital  Readout 

Radar  Altitude  Digital  Readout 

Radar  Altitude  Vertical  Tape 

Radar  Altitude  Vertical  Scale 

Radar  Altitude  High-Low  Alerts 

Rate  of  Climb  Indicator 

Horizon  Line 

N 

N 

Skid/Slip  Ball  &  Lubber  Lines 

3.  Heading/Navigation 

Heading  Scale  &  Lubber  Line 

Command  Heading 

Alternate  Sensor  Bearing 
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Table  3.2  (Continued) 

Apache  Symbology  Changes  from  Mode  Switching 


4.  Central  Cueing/Reference 

Head  Tracker 

Cueing  Dots 

P 

P 

P 

P 

Cued  LOS  Reticle 

P 

P 

P 

P 

5.  Peripheral  Cueing/Reference 

Field  of  Regard  Box 

Field  of  View  Box 

Cued  LOS  Dot 

6.  Weapons  Usage 

Rocket  Steering  Cursor 

P 

P 

P 

P 

Missile  Constraints  Box 

P 

P 

P 

P 

7.  Status  Displays 

Sight  Status 

P 

P 

P 

P 

Weapon  Status 

P 

P 

P 

P 

Weapon  Control 

P 

P 

P 

P 

Range /Range  Source 

P 

P 

P 

P 

N  =  Not  used  in  this  mode 

P  =  Potentially  used,  but  not  controlled  by  mode  switch 


So,  for  the  Apache,  the  symbology  moding  system  is  actually  very  simple  and 
there  are  only  three  cases  in  which  intelligent  moding  rules  would  need  to  be 
devised  (considering  only  the  currently  existing  symbol  set): 

(1)  The  Hover  Position  Box  (On  -  Off) 

(2)  Horizon  Line  (On  -  Off) 

(3)  Velocity  Vector /Acceleration  cue  (60  Kt,  6  Kt,  Off) 

On  the  other  hand,  many  additional  symbols  were  identified  during  this 
project  that  would  require  intelligent  moding  of  some  type. 

Comanche  symbology  modes,  cues,  and  symbols.  Examples  of  the  Comanche 
NOE  and  Cruise  modes  are  shown  in  Figures  3-7  and  3-8  below. 
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Figure  3-7.  Example  of  the  Comanche  NOE  Normal  Mode. 


Figure  3-8.  Example  of  the  Comanche  Cruise  Normal  Mode. 
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Table  3.3,  below  is  provided  to  clarify  the  changes  in  the  Comanche  symbology 
resulting  from  selections  of  the  NOE,  Cruise,  and  Declutter  options.  The  Declutter 
mode  may  be  used  in  injunction  with  either  the  NOE  or  Cruise  modes. 


Table  3.3 

Comanche  Symbology  Changes  from  Mode  Switching 


RAH-66  Comanche 

1.  Position  and  Movement 

Aircraft  Symbol 
Acceleration  Ball 
Airspeed  Indicator  &  VNE  Alert 
Flight  Path  Vector 
Ground  Velocity  Symbol  (High) 
Ground  Velocity  Symbol  (Low) 
Alignment  Dots  for  Hover  Hold 
Hover  Hold  Cue 
Maneuver  Envelope  Cue 
Overspeed  Envelope  Cue 
Hover  Point  Designator 

2.  Attitude/Altitude 
Torque  Indicator  (Normal) 
Torque  Indicator  (Split/Fail) 
Overtorque  Envelope  Cue 
Barometric  Altitude  Indicator 
Radar  Altitude  Digital  &  Analog 
Instantaneous  Vertical  Velocity 
Indicator 

Horizon  Line 
Lateral  Acceleration  Cue 
Pitch  Ladder 

3.  Heading/Navigation 

Heading  Tape  &  Pointer 
Heading  Box  (Digital) 

Current  Waypoint  &  Distance 
Next  Waypoint 
Steering  Cue 
Wind  Indicator 

4.  Central  Cueing/Reference 

N/A 


Modes 

Cruise 

NOE  Declutter 

N 

0-20  kts 

N 

>10  kts 

>  10  kts 

>  10  kts 

N 

20-40  kts 

N 

N 

0-20  kts 

N 

N 

0-40  kts 

N 

N 

vel  stab  on 

N 

C 

C 

C 

C 

C 

C 

N 

command 

N 

C 

C 

C 

C 

C 

C 

N  if <500 

<500  ft 

<500  ft  <500  ft  (dig) 

>  500  ft 

>500  ft 

>40  kts  >40  kts 

>40  kts 

N 

N 

N 


N 

N 
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Table  3.3  (Continued) 

Comanche  Symbology  Changes  from  Mode  Switching 

5.  Peripheral  Cueing/Reference 

Canopy  Rails  C  C  C 

6.  Weapons  Usage 

N/A 

7.  Status  Displays 

Overtemp  Envelope  Cue  C  C  C 

Rotor  Speed  Indicator  C  C  C 

N  =  Not  used  in  this  mode 
C  =  Used  for  special  condition  (not  moded) 

Although  the  Comanche  symbology  scheme  may  appear  to  be  complex,  it  uses 
manual  moding  to  change  the  presentation  status  of  only  six  symbols  of  the  approx¬ 
imately  29  symbols  between  the  Cruise  and  the  NOE  mode.  The  Declutter  mode 
deletes  the  same  six  plus  three  additional  symbols.  A  special  command  is  issued  for 
the  hover  position  designator  box  (depressing  the  boresight  switch  on  the  collective 
while  at  a  hover),  thus  replacing  the  fourth  (Bob-up)  mode  used  on  the  Apache. 

The  Comanche  scheme  is  notable  for  the  two  ways  in  which  intelligent, 
automatic  changes  are  made  to  the  symbology  set  being  displayed.  First,  when  the 
Automatic  mode  is  selected,  the  aircraft  speed  is  used  to  control  changes  between 
Cruise  and  NOE  modes.  The  pilot  may  override  this  control  by  manually  selecting 
Cruise,  NOE,  or  Declutter  modes.  Second,  symbols  may  appear  or  be  deleted  within 
modes  based  upon  airspeed  or  altitude  changes,  or  in  the  advent  of  special  sensed 
conditions,  such  as  exceeding  the  aircraft  envelope. 

ANVIS/HUD  symbology  modes,  cues,  and  symbols.  The  Army  has  recently 
fielded  a  night  vision  goggle  system  with  inset  flight  symbology,  called  the 
ANVIS/HUD.  Approximately  2600  units  will  be  fielded,  using  a  stroked  CRT  with  a 
34°  diagonal  field  of  view,  2:1  contrast  ratio  at  full  moon  (0.005  fL),  and  a  32-step 
brightness  control  (0-  .51  fL).  An  extensive  survey  has  been  performed  in  1996  using 
61  UH-60A/L  pilots  (Piccione,  1996).  The  pilots  rated  the  adequacy  of  the  device  on  a 
number  of  characteristics  and  also  indicated  the  ways  in  which  they  programmed 
the  eight  different  moding  "pages"  to  designate  which  of  the  29  symbols  should 
appear  when  each  page  is  selected  with  a  thumb  switch.  There  are  apparently  no 
reports  of  unit  SOP  for  standardization  of  the  page  content. 

It  is  interesting  to  note  that  most  pilots  did  not  use  all  of  the  modes  available  to 
them.  Of  the  four  Normal  pages,  20%  had  programmed  only  one  page,  49%  two 
pages,  15%  three  pages,  and  13%  four  pages.  Of  the  four  declutter  pages,  30%  had 
programmed  only  one  page,  30%  two  pages,  3%  3  pages,  8%  four  pages,  and  26% 
none  (used  the  default  declutter  mode).  Figure  3-9  shows  the  complete  set  of 
ANVIS/HUD  symbols,  and  Table  3.4  shows  that  the  only  restriction  to  the  moding 
arrangements  are  the  non-deletable  symbols 
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Symbol  Reference  Numbers 


1 .  Angle  of  pitch  scale 

2.  Bearing  to  waypoint  -  pointer 

3.  Compass  reference  scale 

4.  Aircraft  heading  index 

5.  Angle  of  roll  -  pointer 

6.  Angle  of  roll  -  scale 

7.  Barometric  altitude 

8.  Adjust/program  mode  message 

9.  OK/Fail 

10.  Velocity  vector 

1 1 .  Rate  of  climb  pointer 

12.  Radar  altitude  (AGL)  numeric 

13.  Minimum  altitude  warning 

14.  Radar  altitude  (AGL)  analog  bar 

15.  AGL  and  vertical  speed  -  scale 


16.  HUD  fail  message 

17.  Trim  ball 

18.  MST,MEM,HOOK  message 

19.  Sensor,  fire.  RPM  warning 

20.  Horizon  line 

21 .  Display  mode  number 

22.  Torque  limits 

23.  Torque  -  numeric 

24.  Ground  speed 

25.  Indicated  airspeed 

26.  Attitude  reference  indicator 

27.  Engine  temperature 

28.  Distance  to  waypoint 

29.  Bearing  to  waypoint  -  numeric 


Figure  3-9.  Example  of  the  symbols  available  on  the  ANVIS/HUD. 
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Table  3.4 

ANVIS/HUD  Symbology  Changes  from  Mode  Switching 

AN  VI  S/HUD  Normal  Pages  Declutter  Pages 

1.  Position  and  Movement 

Velocity  Vector 
Ground  Speed 

Indicated  Airspeed  ND 

2.  Attitude/Altitude 

Torque  Numeric  &  Limits  Box  ND 

Radar  Altitude  (AGL)  Numeric 
Radar  Altitude  (AGL)  Analog  Bar 


Minimum  Altitude  Warning 
AGL  and  Vertical  Speed  Scale 
Rate  of  Climb  Pointer 

Horizon  Line  ND 

Trim  Ball  &  Lubber  Lines  ND 

Barometric  Altitude  ND 

Attitude  Reference  Indicator  ND 

Angle  of  Pitch  Scale  ND 

Angle  of  Roll  Scale  and  Pointer  ND 

3.  Heading/Navigation 


Compass  Reference  Scale  &  Index 
Bearing  to  Waypoint  Pointer 
Bearing  to  Waypoint  Numeric 
Distance  to  Waypoint 

4.  Central  Cueing/Reference 

None 

5.  Peripheral  Cueing/Reference 

None 

6.  Weapons  Usage 

None 

7.  Status  Displays 


Adjust/Program  Mode  Message  ND  ND 

OK/Fail  ND  ND 

HUD  Fail  Message  ND  ND 

MST,  MEM,  HOOK  Message  ND  ND 

Sensor,  Fire,  RPM  Warning  ND  ND 

Display  Mode  Number  ND  ND 

Engine  Temperature 
ND  =  Not  Deletable 


Otherwise,  moding  of  4  Normal  and  4  Declutter  pages  is  programmed  by  pilot. 


The  decision  of  most  pilots  to  program  no  more  than  two  or  three  pages  is  not 
particularly  surprising  since  it  would  probably  be  quite  difficult  to  remember  the 
contents  of  eight  different  pages.  The  memory  recall  burden  is  worsened  somewhat 
by  the  disappearance  of  the  display  mode  advisory  label  a  few  seconds  after  the  mode 
is  selected.  After  this  brief  period,  relabeling  can  only  be  achieved  by  changing  the 
mode  and  changing  it  back. 

Studying  Modelessness 

Our  objective  was  to  establish  the  utility  of  an  intelligent  symbology 
management  system  by  evaluating  the  capabilities  of  the  advanced  technology  that 
are  available  to  provide  critical  information  in  a  timely  manner  without  developing 
more  complicated  logical  structures  that  the  pilot  must  remember.  This  technology 
must  respond  both  to  operator  input  as  well  as  to  situational  and  system  factors 
without  producing  a  complex  system  of  multiple  modes  that  could  prevent  the  pilot 
from  predicting  and  tracking  mode  transitions  that  he  himself  has  not  initiated. 

We  have  come  to  believe  that  the  most  straightforward  solutions  are  to  reduce 
the  number  of  the  modes,  minimize  their  complexity,  and  provide  clear  feedback  as 
to  their  status.  Providing  appropriate  and  salient  feedback  is  particularly  critical,  so 
that  awareness  of  the  current  situation  can  be  maintained  without  resorting  to 
retrieval  of  system  rules  from  the  pilot's  long-term  memory.  We  chose  to  explore 
the  consequences  of  mode  reduction  or  elimination  by  analysis  of  the  information 
requirements,  examining  each  of  the  hundreds  of  information  elements  we  have 
identified  (e.g.,  airspeed,  barometric  altitude,  current  heading,  etc.)  to  determine 
their  potential  utility  in  the  HMD  symbol  set.  We  reviewed  extensive  task  analysis 
and  timeline  data  to  establish  moment-by-moment  requirements  and  define 
opportunities  for  intelligent  presentation  of  individual  symbols,  for  subsequent 
review  by  Apache  SMEs. 

The  recognition  of  these  intelligent  symbology  management  opportunities  was 
partly  based  on  our  analyses  of  the  cognitive  requirements  of  the  pilot's  tasks.  As  an 
example  of  a  cognitive  requirement  suggesting  an  intelligent  symbol  presentation, 
consider  the  potentially  dangerous  case  of  the  confusion  between  the  meanings  of 
the  command  heading  symbol  in  the  IHADSS  Bob-up  mode  versus  the  other  three 
modes.  Currently,  the  pilot  must  remember  both  the  meaning  of  the  command 
heading  symbol  in  the  Bob-up  mode  (an  indicant  of  aircraft  heading  at  mode 
initiation)  and  the  mode  currently  in  effect,  in  order  to  interpret  the  content  of  the 
symbol.  Appropriate  feedback  of  present  mode  and  information  element  meaning 
could  be  provided  simply  by  using  a  symbol  that  is  distinctly  different  from  the 
command  heading  caret  to  indicate  initial  heading  for  bob-up,  thus  reducing  the 
cognitive  burden  on  the  pilot. 

This  example  is  also  useful  because  it  helps  to  show  that  the  addition  of 
individual  elements  of  symbology  either  automatically,  based  on  system  or 
situational  inputs,  or  the  direct  inputs  of  the  pilot,  can  be  performed  effectively 
without  necessarily  leading  to  complexity  or  pilot  confusions.  Thus,  our  approach 


3-25 


was  to  first  identify  the  individual  information  elements  that  could  meaningfully 
be  presented  on  the  HMD,  and  second  to  determine  whether  they  could  be  made  to 
appear  and  disappear  in  an  intelligent  fashion.  Third,  we  have  attempted  to 
identify  the  specific  system,  situation,  or  pilot-action  cues  that  permit  the  derivation 
of  rules  for  information  element  presentation  or  decluttering.  The  prior  approach 
with  several  sets  of  symbols  in  different  modes  selected  by  the  pilot  could  then  be 
replaced  by  "intelligently  moded"  symbols  individually  controlled  by  a  rule-based 
process.  Thus,  intelligently  moded  symbols  lead  to  system  modelessness  and 
decreased  pilot  workload. 
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Section  4:  Operational  Problems  and  the  HMD 


This  section  of  the  report  is  intended  to  describe  the  broad  range  of  symbols 
potentially  presentable  by  the  HMD.  In  order  to  do  so,  we  have  provided  a  brief  review 
of  Army  mission  management  and  communication  tasks.  This  review  also  serves  as  a 
reminder  of  the  extraordinary  requirements  made  on  military  helicopter  pilots  in 
combat  situations  and  the  need  to  alleviate  as  much  of  this  burden  as  possible. 

Although  task  analyses  and  information  requirements  analyses  are  vital  to  identi¬ 
fying  high-payoff  symbology,  their  dispassionate,  systematic  approach  tends  to  present 
a  stale  and  lifeless  impression  of  what  is  often  a  grueling  cockpit  environment.  In  such 
a  high-workload  environment,  a  primary  objective  of  the  HMD  symbology  designer 
should  be  to  present  critical  data  when  and  where  it  is  needed.  One  of  the  key 
messages  of  this  section  of  the  report  is  that  the  "critical  data"  may  well  be  mission  data 
pertaining  to  positions  in  the  real  world  terrain,  and  not  just  flight  control  information. 

It  is  important  that  symbology  designers  keep  in  mind  that  responding  to  an  actual 
tactical  situation  is  considerably  more  demanding  than  simply  flying  the  aircraft, 
viewing  sensor  data,  and  operating  various  systems.  With  little  warning  and  with  great 
urgency,  the  speed  and  flexibility  of  attack  and  scout  helicopters  can  be  called  upon  to 
rapidly  perform  diverse  operations  such  as: 

•  Concentrate,  disperse,  or  redeploy  to  extend  the  area  of  influence. 

•  React  to  tactical  opportunities  and  necessities. 

•  Conduct  exploitation  and  pursuit  operations. 

•  Place  forces  at  tactically  decisive  points  in  the  battle  area. 

•  React  to  threats  in  friendly  rear  areas. 

•  Secure  and  defend  key  terrain  or  deep  objectives. 

•  Bypass  enemy  positions  and  achieve  surprise. 

•  Conduct  fast-paced  operations  over  extended  distances. 

Army  Aviation  Mission  Management 

In  identifying  the  types  of  shortcomings  military  helicopter  pilots  routinely  face  in 
operational  settings,  it  becomes  apparent  that  the  most  pervasive  problems  pertain  to 
the  quality  of  the  information  provided  to  the  pilot  for  conduct  of  his  combat  data 
handling,  or  "mission  management"  tasks.  Although  basic  capability  enhancements 
such  as  in  aircraft  speed  or  weapon  accuracy  are  often  suggested  to  ease  the  pilots 
burden,  most  of  the  workload  problems  of  combat  operations  are  imposed  by  the 
enormous  demands  of  the  mission  management  functions.  These  operational  problems 
include  difficulties  in  both  information  acquisition  and  information  interpretation. 
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To  illustrate  the  nature  of  the  mission  management  functions,  it  is  useful  to  consider  the 
typical  roles  of  the  attack  helicopter  company  commander  who: 

(1)  Receives  the  mission  from  the  commander  or  the  S3  (ATKHB  or  ground 
unit) 

(2)  Provides  detailed  planning  guidance  to  crews  of  company  helicopters 

(3)  Coordinates  with  the  ground  unit  operating  in  the  battle  area 

(4)  Selects  primary  and  subsequent  battle  positions  (BPs)  for  attack  helicopters 

(5)  Plans  routes  to  holding  areas  (HAs)  and  BPs 

(6)  Coordinates  indirect  fire  support  and  close  air  support  (CAS) 

(7)  Arranges  joint  air  attack  team  (JAAT)  operations 

(8)  Controls  attack  helicopter  fire  in  BPs 

(9)  Keeps  his  battalion  commander  continually  informed  of  the  situation 

(10)  Plans  for  and  provides  local  security  for  attack  helicopters 

(11)  Briefs  the  incoming  ATKHC  commander  during  relief-on-station  operations 

(12)  Maneuvers  aircraft  to  FARPs 

(13)  Directs  rearming  and  refueling  operations 

(14)  Conducts  detailed  debriefings 

(15)  Prepares  for  the  next  mission 

During  prior  Anacapa  studies  of  Army  aviation  roles  and  responsibilities,  it  has 
become  clear  that  there  is  a  general  consensus  among  Army  aviators  that  the  attack 
helicopter  company  (ATKHC)  company  commander  is  "the  busiest  crew  member  on  the 
battlefield."  Nevertheless,  similarly  demanding  types  of  mission  management  functions 
are  performed  by  the  subordinate  attack  team  leaders  and  to  some  degree  by  all  attack 
and  reconnaissance  helicopter  pilots. 

There  are  other  types  of  high-workload  activities  that  are  best  represented  by  the 
armed  reconnaissance  mission,  and  planned  by  the  air  cavalry  troop  (ACT)  commander, 
whose  overall  list  of  responsibilities  is  similar  to  that  of  the  ATKHC  commander. 
Furthermore,  the  ACT  commander  is  expected  to  be  extremely  flexible  in  planning  and 
management  of  assigned  missions  while  airborne  and  in  gathering  and  transmitting 
situation  data  for  use  in  the  continuous  planning  and  management  processes. 

Because  the  primary  mission  of  the  air  cavalry  is  reconnaissance,  cavalry  units  are 
more  likely  than  other  aviation  units  to  fly  in  areas  where  the  tactical  situation  is 
unknown  or  unclear.  Thus,  although  air  cavalry  unit  leaders  attempt  a  thorough  map 
study  before  flight,  they  may  be  given  little  information  regarding  the  enemy  situation 
and  must  concentrate  on  terrain  analysis  and  the  potential  for  enemy  movements.  Fight 
route  study  and  selection  is  a  never-ending  activity  for  air  cavalry  aviators  because  their 
mission  seldom  requires  that  they  select  a  "best"  route,  but  instead  that  they  seek  out 
enemy  activity  on  all  potential  routes  and  areas. 
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The  Evolving  Implications  of  Mission  Management 

Military  helicopters  are  best  employed  in  combined  arms  tactics  that  require 
surprise,  flexibility,  mobility,  speed,  and  rapid  shifting  or  concentration  of  combat 
power.  These  capabilities  are  enjoyed  to  the  same  degree  by  no  other  forces,  and  enable 
the  commander  to  extend  the  depth  and  width  of  the  battlefield,  and  to  gain  and 
maintain  the  initiative.  These  tactics  and  capabilities,  however,  lead  directly  to  a 
requirement  for  rapid  mission  management  activities.  It  is  imperative  that  mission 
management  activities  be  conducted  quickly  and  accurately  in  order  to  ensure  that  the 
speed  and  flexibility  of  Army  aviation  units  can  be  invoked  to  take  advantage  of 
momentary  opportunities  in  the  tactical  situation. 

Furthermore,  given  the  large  geographical  area  in  which  military  helicopters  may 
operate,  a  great  deal  of  precious  time  is  consumed  if  company  and  troop  commanders 
must  return  to  a  tactical  operations  center  (TOC)  to  confer  with  the  commander  and 
staff  officers  before  commencing  a  new  mission.  Thus,  it  is  clear  that  aviators  will  often 
be  required  to  perform  mission  management  in  the  helicopter  cockpit — either  in  a 
concealed  position  on  the  ground,  or  while  enroute  to  the  new  mission  location. 

It  is  also  important  to  recognize  that  mission  management  is  a  continuous,  rather 
than  an  intermittent,  task.  In  order  to  simplify  the  complex  management  process  for 
training  purposes,  field  manuals  tend  to  emphasize  the  development  of  operations 
orders  (OPORDs)  through  face-to-face  planning  sessions,  and  to  describe  the  distribu¬ 
tion  of  paper  copies  of  operations  orders  along  with  various  map  overlays. 

What  is  missing  from  these  characterizations  is  that  once  the  operations  have 
begun,  the  planning  efforts  do  not  cease,  but  proceed  continuously.  Situation  reports 
describing  the  shifting  battlefield  situation,  and  fragmentary  orders  (FRAGOs)  issued  to 
adjust  the  actions  of  the  friendly  forces  are  frequently  transmitted  from,  and  received  in, 
helicopter  cockpits.  To  this  extent,  on-board  mission  management  has  already  become 
one  of  the  military  helicopter  pilot's  required  functions. 

The  following  subsections  of  the  report  describe  (a)  the  special  operational 
problems  currently  experienced  in  acquiring  timely  tactical  information,  (b)  the  nature 
of  the  requirements  for  the  flow  of  tactical  operation,  and  (c)  the  difficulties  in 
extracting  or  applying  this  information  from  current  materials.  Following  these 
descriptive  examples  are  some  potential  applications  of  advanced  communication,  map, 
and  HMD  systems  to  overcome  these  shortcomings. 

An  Overview  of  Communication  Requirements 

The  rapidity  with  which  Army  aviation  operates,  and  the  great  distances  over 
which  it  may  extend  its  resources,  require  that  special  attention  be  given  to  the  commu¬ 
nication  systems  that  enable  command  and  control  of  these  operations.  It  is  absolutely 
imperative  that  such  systems  function  quickly  and  effectively  in  order  to  ensure  that  the 
speed  and  flexibility  of  Army  aviation  units  can  be  invoked  in  time  to  take  advantage  of 
momentary  opportunities  in  the  tactical  situation. 
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Effective  mission  management  is  dependent  upon  the  availability  of  current  tacti¬ 
cal  information.  Maintaining  the  flow  of  tactical  information  is  especially  difficult  for 
Army  aviation  units  because  of  the  units'  large  areas  of  responsibility,  rapid-response 
requirements,  and  the  wide  variety  of  combat  roles  they  may  be  fulfilling. 

The  arteries  and  veins  of  the  tactical  information  distribution  system  are  its  orders 
and  reports.  Processed  information  is  carried  downward  through  the  echelons  by 
means  of  orders,  and  raw  battlefield  information  is  passed  upward  by  means  of  reports. 
The  processed  information  passed  downward  is  an  analysis  of  battlefield  information 
from  many  sources,  integrated  to  provide  a  relatively  complete  picture  of  the  strength, 
intent,  and  location  of  enemy  forces,  as  well  as  the  locations  and  planned  actions  of 
friendly  units. 

In  contrast,  the  battlefield  information  passed  upwards  through  echelons  is 
relatively  unprocessed  data  consisting  primarily  of  spot  reports  of  observed  enemy 
activity  and  situation  reports  describing  circumstances  on  the  battlefield.  Report  data 
are  not  integrated,  but  have  the  advantage  of  being  the  most  current  available. 

Orders.  One  of  the  major  functions  of  the  company  commander  is  that  of 
receiving  and  interpreting  combat  orders — warning  orders,  operation  orders 
(OPORDs),  and  fragmentary  orders  (FRAGOs).  The  OPORD  format,  which  is  the  same 
at  every  echelon,  is  composed  of  five  paragraphs:  (1)  situation,  (2)  mission, 

(3)  execution,  (4)  service  support,  and  (5)  command  and  signal.  A  full,  written  OPORD 
at  battalion  level  may  extend  to  20  typed  pages  or  longer,  and  is  usually  accompanied 
by  a  series  of  annexes  such  as  the  following: 

•  Intelligence  •  Fire  Support 

•  Operations  Overlay  •  Operations  Schedule 

•  Air  Movement  •  Operations  Security 

•  Air  Space  Management  •  Service  Support  (Logistics) 

•  Air  Defense  •  Communications-Electronics 

Only  one  of  these  ten  annexes  is  described  as  an  overlay,  but  in  fact  each  annex 
may  have  a  series  of  appendices  that  include  overlays.  For  example,  the  air  movement 
annex  will  normally  include  one  or  more  overlays  (sometimes  for  use  with  different 
map  scales)  including  air  corridor  information  such  as  shown  in  Figure  4-1.  The  fire 
support  annex  will  be  accompanied  by  a  preplanned  target  overlay  such  as  shown  in 
Figure  4-2,  to  simplify  calling  fire  upon  observed  enemy  activity. 

In  addition  to  the  point  targets  of  the  preplanned  target  overlay,  a  number  of  other 
graphics  may  be  used  to  restrict  or  control  fires.  Figure  4-3  shows  examples  of  a  fire 
support  coordination  line,  a  coordinated  fire  line,  a  restricted  fire  line,  a  free-fire  area,  a 
no-fire  area,  and  a  restricted  fire  area.  Even  from  these  simple  examples,  it  is  evident 
that  the  shapes  of  the  lines  and  areas  typically  are  not  possible  to  describe  by  a  voice  or 
text  messages. 
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Air  corridor  with  lateral  limits  --  moderately  stringent 


Air  corridor  with  no  lateral  diversions  --  most  stringent 
Figure  4-1.  Three  methods  of  indicating  air  corridors  on  a  tactical  overlay. 


The  fire  support  annex  will  almost  always  include  a  preplanned  target  overlay,  as 
shown  in  Figure  4-2. 
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Figure  4-3.  Samples  of  graphics  used  to  restrict  or  control  fire. 

In  addition,  the  intelligence,  air  movement,  air  space  management,  air  defense, 
logistics  and  communications-electronics  annexes  often  include  map  overlays.  The 
reason  map  overlays  are  so  frequently  used  in  conjunction  with  orders  is  that  an  enor¬ 
mous  amount  of  spatial/ geographic  information  must  be  conveyed  in  order  to  conduct 
operations. 

In  order  to  control  the  various  elements  under  his  command,  the  attack  helicopter 
company  commander  would  draw  his  own  overlay  for  distribution  to  his  subordinates, 
carefully  illustrating  his  concept  of  the  operation,  boundaries,  phase  lines,  hazards  to 
flight,  ACPs,  HAs,  BPs,  and  EAs.  An  example  of  such  an  overlay,  drawn  by  a  SME 
during  an  Anacapa  knowledge  acquisition  session,  is  shown  in  Figure  4-4.  This  overlay 
is  intended  for  use  on  a  l:50,000-scale  map.  Thus,  one  centimeter  equals  500  meters  and 
one  inch  equals  1389  yards. 

There  are  several  important  points  to  be  recognized  from  this  brief  discussion  of 
operations  orders.  First,  it  is  clear  that  graphics  are  used  in  operations  orders  because 
they  most  effectively  and  efficiently  convey  extensive  and  critical  battlefield  informa¬ 
tion.  Second,  operations  orders,  because  of  their  length  and  complexity,  would  almost 
never  be  transmitted  over  the  radio.  Third,  when  the  tactical  situation  does  not  permit 
face-to-face  briefings  (and  the  distribution  of  operations  orders,  annexes,  and  overlays), 
but  instead  requires  the  use  of  oral  FRAGOs  over  the  radio,  there  is  a  very  high 
probability  of  encountering  difficulties  in  communicating  the  critical 
spatial /geographic  information. 

Reports.  Just  as  the  commander  must  be  able  to  disseminate  orders  without  delay 
or  confusion,  so  must  he  be  able  to  receive  continuous,  reliable,  and  current  combat 
information  about  the  friendly  and  enemy  situation  if  he  is  to  win  the  battle.  The  spot 
report  is  used  to  convey  information  about  the  enemy  and  his  operations  and  is  ordered 
by  the  acronym  "salute,"  for:  size,  activity,  location,  unit,  time,  and  equipment.  A  spot 
report  is  as  brief  as  possible,  as  in:  "Ten  T-62  tanks  moving  south  grid  Sierra  Tango  four 
one  six  two  three  seven  at  one  four  two  zero  zulu." 

The  portion  of  the  report  stating  "grid  Sierra  Tango  four  one  six  two  three  seven" 
is  the  geographic  position  of  the  tanks,  expressed  as  in  Universal  Transverse  Mercator 
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Figure  4-4.  Examples  of  map  overlay  annotations  used  by  an  attack  helicopter  company. 
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(UTM)  grid  coordinates.  Although  the  UTM  grid  system  permits  location  of  a  point 
within  10  meters,  use  of  the  system  to  this  level  of  accuracy  requires  that  a  plastic  coor¬ 
dinate  scale  be  overlayed  on  the  map.  The  recipient  of  the  coordinates  must  also  use  the 
coordinate  scale  in  order  to  plot  the  point  on  the  map.  Because  the  coordinate  scale  is 
unsuitable  for  use  in  flight,  aviators  almost  never  attempt  to  estimate  positions  closer 
than  100  meters,  that  is,  in  6-digit  grid  coordinates.  Even  at  this  level  of  accuracy,  it  is 
commonly  acknowledged  that  errors  are  made  in  determining  the  coordinates,  commu¬ 
nicating  these  coordinates,  and  in  applying  them  to  locate  a  point  on  the  map.  Of 
course,  accurate  coordinate  identification  is  useless  if  the  pilot  has  mistakenly  identified 
a  landform  on  the  map  as  representing  one  he  sees  in  the  real  world. 

When  battle  damage  assessments  (BDAs)  follow  spot  reports,  additional 
spatial  /geographic  data  are  not  required  unless  the  speed  and  direction  of  escaped 
enemy  vehicles  is  transmitted.  Spot  reports  and  BDAs  are  used  primarily  at  lower 
echelons,  such  as  between  the  company  commander  and  the  battalion  S-3. 

Situation  reports  are  used  at  all  echelons,  and  are  more  extensive  than  spot  reports. 
They  may  provide  more  complex  geographical  data,  such  as  the  movement  and 
changing  shape  of  an  area  of  engagement,  changes  in  the  FLOT,  and  so  forth.  Situation 
reports  are  especially  important  when  the  situation  is  not  developing  as  anticipated 
during  planning,  as  in  the  cases  of  unexpected  enemy  troops  or  under-predicted  speed 
of  movement.  In  such  cases,  the  information  must  be  relayed  to  the  S3s  as  quickly  and 
accurately  as  possible  for  use  by  the  commanders. 

Problems  of  Communicating  Required  Spatial/Geographic  Data 

Anacapa  interviews  with  key  command  and  staff  officers  at  the  101st  Division 
(Rogers  &  Spiker,  1987)  have  revealed  that  there  are  often  problems  in  rapidly  sharing 
tactical  data  with  all  of  those  who  need  them.  Many  of  the  information  elements  that 
must  be  transmitted  are  spatial  and  geographic,  such  as  planned  routes  or  free-fire 
areas.  When  possible,  these  spatial/ geographic  data  are  obtained  by  tracing  map  over¬ 
lays  at  the  TOC.  One  problem  in  hand-tracing  situation  maps  is  that  errors  of  omission 
and  commission  begin  to  creep  into  the  data,  and  with  time  pressures,  data  such  as 
logistics  information,  adjacent  units,  deep  enemy  forces,  and  friendly  artillery  locations 
may  be  knowingly  omitted,  only  to  be  needed  later. 

Time  Delays.  Time  delays,  however,  are  a  greater  problem.  Because  of  the 
current  delays  in  getting  tactical  information  to  the  supported  unit,  the  time  taken  in 
posting  the  information  to  the  unit  situation  map,  the  time  needed  by  the  messenger  to 
trace  the  map,  and  the  flight  time  returning  to  the  TOC,  the  tactical  information 
received  by  the  BC  could  be  several  hours  old.  Given  the  fluidity  of  the  battlefield 
situation,  tactical  data  this  old  could  be  misleading,  and  could  significantly  diminish  the 
ability  of  the  aviation  unit  to  effectively  take  advantage  of  the  speed  and  flexibility  of  its 
assets.  As  stated  previously.  Army  aviation  is  best  employed  in  tactics  that  require 
surprise,  speed,  and  rapid  shifting  or  concentration  of  combat  power.  Such  tactics  are 
impossible  without  fresh  situation  data. 
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Thus,  it  is  clear  that  transmission  of  situation  data  is  required.  A  great  deal  of  time 
is  lost  if  the  company  commander  is  forced  to  periodically  send  messengers  to  the  TOC 
to  copy  the  situation  map  overlays.  In  addition,  returning  to  the  TOC  may  be  impracti¬ 
cal  or  impossible  within  the  time  constraints  of  the  mission,  and  the  information  must 
be  exchanged  by  voice  radio.  Our  prior  interviews  with  subject-matter  experts  at  the 
101st  Division  confirmed  that  these  problems  have  already  been  experienced  in  tactical 
exercises. 

Even  when  voice  radio  transmissions  are  not  degraded  due  to  large  distances  or 
jamming,  the  exchange  of  spatial/ geographic  data  is  difficult.  Transmission  of  six-  or 
eight-digit  UTM  grid  coordinates  permits  the  exchange  of  some  positional  information, 
but  this  approach  is  slow,  subject  to  frequent  errors,  and  inefficient  in  describing  curved 
lines  or  area  boundaries.  Such  complex  spatial/ geographic  data  as  current  friendly  and 
enemy  situation,  scheme  of  maneuver,  fire  support,  air  routes  and  corridors,  phase 
lines,  boundaries,  and  other  tactical  data  are  very  difficult  to  describe  by  voice  radio 
transmission. 

The  results  of  these  problems  are  reflected  in  a  frequent  inability  to  share  informa¬ 
tion  on  the  positions  of  friendly  and  enemy  units  on  the  battlefield,  as  well  as  in 
problems  getting  specific  tactical  information  items  to  the  key  players  who  need  them. 

The  potential  difficulties  in  communicating  the  spatial/ geographic  information 
critical  to  effective  orders  are  especially  important  because  it  is  anticipated  that  in  future 
battles  there  will  be  limited  opportunities  for  briefings  and  distribution  of  paper  orders 
and  map  overlays.  Instead,  the  oral  FRAGO  will  be  the  rule  rather  than  the  exception, 
and  commanders  can  expect  to  receive  mission  change  orders  without  the  benefit  of 
acetate  overlays  describing  the  current  friendly  and  enemy  situation,  scheme  of 
maneuver,  fire  support,  air  routes  and  corridors,  phase  lines,  boundaries  and  other 
tactical  data. 

Requirements  for  Continuous  Planning.  The  nature  of  the  communications 
problems  described  here  are  not  typically  made  evident  in  Army  field  manuals  and 
training  circulars  because  these  sources  tend  to  emphasize  the  development  of  opera¬ 
tions  orders  through  face-to-face  planning  sessions,  and  the  distribution  of  paper  copies 
of  OPORDS  along  with  various  map  overlays.  In  reality,  once  the  operations  have 
begun,  the  planning  efforts  do  not  cease,  but  proceed  continuously. 

Continuous  planning,  however,  often  precludes  both  the  use  of  face-to-face 
consultations  and  the  distribution  of  tactical  data  in  the  form  of  acetate  or  paper  map 
overlays.  Instead,  radio  transmissions  or  messengers  must  be  used  to  share  tactical  data 
and  issue  FRAGOs.  Because  the  contents  of  a  FRAGO  may  include  any  of  the  data  used 
in  an  OPORD,  it  is  a  certainty  that  overlay-related  information  is  needed  in  these 
communications,  and  it  cannot  often  wait  until  the  next  face-to-face  encounter  at  the 
TOC. 

During  an  operation,  when  the  S3  and  the  BC  are  within  secure  voice  radio  range, 
they  attempt  to  maintain  situation  overlays  by  sending  six-digit  grid  coordinates,  but 
this  is  acknowledged  to  be  a  slow  and  crude  method  of  transmitting  data.  The  commu- 
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nications  electronics  operating  instructions  (CEOI)  message  coding,  authentication 
tables,  and  terrain  index  reference  system  coding  described  in  Army  field  manuals  are 
simply  "time  prohibitive"  and  too  error-prone,  and  are  never  actually  used. 

Large  areas  of  operation.  The  area  of  operations  for  the  attack  helicopter  battalion 
is  very  large — essentially  the  same  as  that  of  the  division.  The  attack  helicopter  compa¬ 
nies  must  be  ready  to  move  anywhere  in  this  area,  crossing  brigade  boundaries  or 
performing  deep  attacks  across  the  FLOT,  with  only  minimal  advance  notice.  It  follows 
that  each  of  the  company  commanders  needs  to  be  furnished  with  the  most  comprehen¬ 
sive  and  current  tactical  overlay  data  available  before  moving  to  the  new  area  of  opera¬ 
tions.  Unfortunately,  this  is  not  currently  the  case.  Furthermore,  attempting  to  obtain 
data  by  moving  to  an  unfamiliar  TOC  area  without  a  clear  definition  of  enroute  dangers 
from  enemy  (as  well  as  friendly)  activities  can  be  risky. 

Preplanned  control  measures  such  as  phase  lines  and  checkpoints  become  very 
important  in  command  and  control  during  the  battle  because  they  permit  the  transmis¬ 
sion  of  spatial-geographic  data  that  are  understood  by  both  communicators.  Unfortu¬ 
nately,  given  such  large  areas  of  operations  it  is  impossible  to  foresee  all  of  the  possible 
zones  in  which  aviation  units  may  operate,  especially  since  the  enemy  actions  are  not 
completely  predictable. 

As  a  result  of  these  uncertainties,  changes  in  the  mission  transmitted  by  FRAGOs 
may  often  place  aviation  units  in  areas  for  which  preplanned  control  measures  are 
unavailable  or  insufficient.  These  situations  are  a  natural  outcome  of  the  rapid 
redeployment  advantages  of  Army  aviation,  and  should  be  expected  to  be  a  frequent 
occurrence  in  future  conflicts.  As  a  result,  the  BC  or  his  messenger  must  either  fly  to  the 
ground  maneuver  TOC  to  examine  the  local  situation  map  and  trace  the  pertinent 
overlays  or  attempt  to  overcome  the  clear  deficiencies  of  voice  transmission  of  spatial 
data.  An  ability  to  transmit  new  phase  lines,  sectors,  fire  control  data,  air  corridors, 
routes,  holding  areas,  and  so  forth  will  become  not  only  a  convenience,  but  a  necessity 
for  conduct  of  fast-paced  operations  over  extended  distances. 

Emerging  Solutions  to  Mission  Management  Problems 

In  future  rotorcraft,  innovative  technology  will  be  incorporated  to  vastly  improve 
mission-management  efficiency.  There  are  three  broad  technology  categories  that 
address  the  special  operational  problems  described  in  this  section  of  the  report.  They 
include  better  tactical  communication  systems  to  ensure  the  necessary  data  is  available; 
advanced  digital  map  systems  to  provide  more  compatible  views  of  the  terrain  and 
intervisibility  computations  for  determining  masking;  and  HMD  systems  and  symbol¬ 
ogy  to  facilitate  the  integration  of  the  extensive  spatial /geographic  data  in  the  most 
intuitive  and  helpful  manner.  The  following  pages  present  brief  summaries  of  the  new 
communications  and  digital  map  contributions,  followed  by  a  more  extensive  descrip¬ 
tion  of  the  HMD  contributions. 

Communications  Solutions  to  Mission  Management  Problems.  In  the  near 
future,  powerful  improvements  in  radio  systems  and  data  link  capabilities  will  be 
implemented  permitting  the  storage  and  transmission  of  extensive  amounts  of  tactical 
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information.  These  systems  will  permit  secure,  jam-resistant  communications  that  will 
not  only  reduce  cockpit  workload,  but  will  eliminate  many  of  the  current  requirements 
for  flights  to  the  TOC  to  deliver  combat  reports  or  to  trace  the  situation  overlays. 
Furthermore,  the  new  systems  will  permit  "databurst"  transmissions  among  aircraft  to 
share  tactical  data  without  recourse  to  voice  transmissions  or  actual  landings  for  confer¬ 
ences  over  paper  maps  and  acetate  overlays. 

For  example,  the  RAH-66  Comanche  will  support  data  linking  through 
implementation  of  a  high-density  data  transfer  module  that  will  permit  the  storage  and 
exchange  of  huge  amounts  of  mission-planning  data,  both  on  the  ground  and  in  the  air. 
The  Data  Transfer  System  on  the  Comanche  will  provide  automatic  loading  of  such 
data  bases  as  terrain,  CEOI,  meteorological,  and  C3I.  The  Comanche  will  also  receive  a 
Level  III  integration  of  the  1553B  data  bus  that  will  support  data  transmission  at  rates  50 
times  greater  than  the  current  B  version.  This  has  the  potential  to  permit  fusion  of  EO 
sensors,  RF  sensor  inputs,  aided  target  recognition,  automatic  sector  search  for 
improved  target  acquisition,  and  inflight  reconfiguration.  It  also  supports  further 
modularization  and  miniaturization  of  avionics  systems. 

The  improvements  in  tactical  communications  systems  will  permit  data-bursting 
the  required  map  overlay  data  including  situation  graphics,  fire  control  data,  and  other 
geographic  position  information  to  the  cockpit.  In  return,  the  pilot  will  be  able  to 
immediately  transmit  the  most  current  battlefield  report  information  to  his  commander. 

The  key  issue  is  that  these  new  capabilities  will  pave  the  way  for  a  real-time  situa¬ 
tion  awareness  capability  in  the  cockpit  that  is  so  strongly  desired  by  aviators.  For 
example,  the  data  from  new  FAADS  sensors  that  are  designed  to  identify  and  locate 
enemy  helicopters  could  be  sent  directly  to  friendly  aircraft,  instead  of  through  the 
TOC.  In  a  related  application,  targets  could  be  rapidly  passed  to  direct  support  (DS) 
artillery,  and  some  safety  provided  by  passing  friendly  helicopter  positions  at  the  same 
time. 

Another  example  of  a  likely  real-time  situation  awareness  capability  will  be  a 
method  of  avoiding  enemy  air  defense  artillery  (ADA)  engagement  areas.  Enemy  ADA 
sensors  are  usually  turned  off  and  do  not  begin  emitting  until  friendly  helicopters  get 
close,  so  there  is  minimal  preparation  time  for  dealing  with  this  danger.  With  a  real¬ 
time  threat  display  system,  the  weapon  positions  and  friendly  aircraft  positions  can  be 
relayed  immediately  to  friendly  artillery  for  a  suppression  decision,  and  to  friendly 
helicopters  for  a  decision  to  bypass  or  engage  the  sites.  There  is  no  similar  capability 
currently  available,  and  there  is  concern  that  in  a  conflict  the  aviation  units  would  be,  as 
one  pilot  put  it,  "just  blundering  along,  marking  the  threat  with  our  burning  helicop¬ 
ters." 


Digital  Map  Solutions  to  Mission  Management  Problems.  Just  as  paper  maps 
have  been  the  focal  point  of  planning  and  navigation  in  the  past,  digital  maps  will  form 
the  heart  of  most  future  mission-planning  systems  and  head-down  cockpit  displays, 
integrating  diverse  spatial/ geographic  information.  The  use  of  the  digital  map  display 
has  some  enormous  advantages  over  paper  maps,  especially  if  digital  terrain  elevation 
data  is  provided.  In  the  mission-planning  setting,  the  digital  map  display  facilitates 
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terrain  analysis,  waypoint  and  route  data  handling,  and  special  computations,  and 
offers  a  comprehensive  and  comprehensible  method  of  integrating  important,  complex 
spatial  elements  of  the  plan. 

Digital  maps  are  subdivided  into  two  major  categories:  computer-generated  maps 
and  digitized  paper  maps.  Computer-generated  maps  provide  extraordinary  advan¬ 
tages  for  the  planner.  They  permit  him  to  generate  "tailored"  products,  such  as  maps  in 
any  scale  with  any  subset  of  topographic  features.  Computer-generated  maps  can  show 
the  landform  contours  in  plan  or  perspective  view,  using  slope-shaded  surfaces  to 
convey  a  three-dimensional  appearance  that  is  extremely  useful  in  terrain  analyses.  The 
digitized  paper  map,  in  contrast,  permits  use  of  the  high-resolution  data  available  from 
the  huge  selection  of  existing  maps  and  charts.  Either  computer-generated  maps  or 
digitized  paper  maps  can  be  augmented  with  intervisibility  calculation  displays  based 
on  terrain  elevation  data.  The  primary  advantages  of  both  types  of  digital  maps  are 
summarized  in  the  following  four  paragraphs. 

•  Waypoint  and  Route  Data  Handling — The  painstaking  process  of  determin¬ 
ing  the  coordinates  and  elevation  of  waypoints  and  other  positions  from 
paper  maps  can  be  entirely  eliminated.  Instead,  a  cursor  can  be  positioned 
on  the  map  display  screen,  and  these  data  furnished  instantly.  When  neces¬ 
sary,  latitude  and  longitude  can  be  automatically  converted  to  UTM  coordi¬ 
nates.  The  range  and  bearing  between  waypoints  can  then  be  automatically 
forwarded  to  other  components  of  the  system  for  time /fuel /distance  calcu¬ 
lations.  The  tedious  annotation  of  paper  maps  can  also  be  eliminated,  with 
course  lines  and  waypoint  data  automatically  entered  for  display  on  digital 
maps  in  the  cockpit. 

•  Terrain  Analysis — The  perceptual  task  of  relating  map  contour  lines  to  the 
expected  appearance  of  terrain  relief  in  the  area  of  operations  is  perhaps  the 
most  difficult  aspect  of  map  interpretation.  The  computer-generated  map 
can  be  used  to  present  a  slope-shaded,  plan-view  depiction  of  the  terrain, 
having  the  appearance  of  a'3-dimensional  relief  map,  thereby  making  the 
actual  landform  contours  immediately  apparent  to  the  viewer.  In  a  related 
technique,  the  image  can  be  "rotated"  to  provide  a  perspective  view  of  the 
terrain,  familiarizing  the  pilot  with  the  landforms  as  they  would  be  seen 
during  flight  at  any  chosen  altitude  and  heading. 

•  Intervisibility  Computations — The  digital  map  and  its  associated  terrain  ele¬ 
vation  data  base  can  be  used  to  perform  mission-planning  computations  of 
previously  forbidding  complexity.  Masking  and  backdrop  from  known  or 
suspected  enemy  positions,  fields  of  fire  from  an  aircraft  at  any  given  posi¬ 
tion  and  altitude,  and  other  computations  are  made  simple.  Among  the 
most  critical  of  these  computations  are  the  threat  lethality  envelopes 
surrounding  known  or  suspected  weapon  positions.  Such  positions  are 
currently  shown  on  paper  maps  by  simple  circles  representing  maximum 
weapon  ranges.  Yet,  lethality  envelopes  may  now  be  calculated  to  show  the 
masking  effects  introduced  by  terrain  at  any  selected  aircraft  altitudes. 
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Figure  4-5.  An  example  of  a  slope-shaded  plan  view  digital  map  format. 


Figure  4-6.  An  example  of  a  digital  map  perspective  view  of  terrain. 
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•  Information  Integration — The  digital  map  offers  the  most  direct  and  easily 
understood  method  of  integrating  the  hundreds  of  spatial  and  geographic 
information  elements  dealt  with  during  the  planning  process.  Various  sets 
of  overlays  from  the  intelligence,  weather,  operations,  or  other  data  bases 
may  be  displayed  at  will,  or  rapidly  deleted  to  eliminate  clutter  and  permit 
the  underlying  terrain  to  be  examined.  Special-purpose  annotations  may 
also  be  introduced  to  prepare  the  display  for  inflight  use. 

HMD  Solutions  to  Mission  Management  Problems.  Applications  of  HMDs  in 
advanced  rotorcraft  focus  on  their  ability  to  present  flight  symbology  similar  to  the 
HUD's,  reducing  head-down  time  and  eye  re-focus  time,  and  providing  conformal 
symbology  such  as  airfield  outlines.  The  ability  of  the  HMD  to  be  used  off-axis  is 
assumed  to  improve  the  ability  to  search  the  real-world  environment  while  still  viewing 
critical  flight  data.  Thus,  there  is  a  great  deal  of  discussion  about  such  issues  as  poten¬ 
tial  confusions  judging  aircraft  attitude  from  observing  a  forward  view  of  the  horizon 
line  while  looking  90°  to  the  left  or  right  of  the  aircraft  centerline. 

Overlooked  in  this  conceptualization,  however,  are  the  extraordinary  opportuni¬ 
ties  to  integrate  mission  management  data  in  a  more  intuitive  and  informative  manner 
than  has  ever  before  been  possible.  The  HMD,  integrated  with  spatial-geographic 
information  provided  through  advanced  communication  systems  and  digital  map 
devices  provides  a  long-awaited  method  of  bridging  the  gap  between  the  real  world 
and  its  traditional  symbolic  encodements  (alpha-numeric  data  and  map  icons). 

Expressed  concisely,  given  new  communication  systems  capable  of  rapid 
transmission  of  extensive  graphic  data  and  new  digital  map  systems  capable  of  selec¬ 
tively  displaying  all  manner  of  head-down  tactical  situation  graphics,  HMDs  can.be 
used  for  presentation  of  any  of  these  important  information  elements  in  the  form  of 
symbology  overlaid  on  (conformal  with)  the  real  world  or  on  simulated  perspective 
views  of  the  world  generated  from  digital  terrain  elevation  data. 

In  addition  to  this  "augmented  reality"  capability,  the  HMD  can  be  used  not  only 
as  a  display,  but  as  an  input  device  to  the  new  digital  maps  and  communications 
systems.  By  use  of  manual  controls  or  head  movements,  the  pilot  can  quickly  "point 
out"  positions  in  the  real  world  using  indicators  in  his  HMD  symbology  set.  He  can 
then  cause  these  positions  to  be  highlighted  on  the  maps  or  in  the  HMD  views  of  other 
aircraft  to  give  an  "over-the-shoulder"  immediacy  to  his  orders  and  reports. 

The  HMD  and  Cognitive  Workload  Reduction.  As  an  overview  to  the  utility  of  the 
HMD  in  solving  mission  management  problems,  it  is  useful  to  consider  the  nature  of 
the  pilot's  cognitive  tasks  in  dealing  with  the  real-world  position  of  tactical  information 
elements.  In  particular,  the  HMD's  unique  potential  for  enhancing  the  pilot's  situation 
awareness  is  related  to  the  concept  of  "visual  momentum,"  a  term  which  was  originally 
used  to  describe  the  visual  impetus  or  continuity  a  cinematographer  attempts  to  create 
in  film  editing  to  establish  a  visual  coherence  across  several  scenes  (Hochberg  and 
Brooks,  1978).  The  term  was  later  expanded  by  Woods  (1984)  to  describe  the  features  of 


4-14 


a  system  design  that  enhance  an  operator’s  ability  to  extract  and  integrate  information 
across  several  displays. 

A  greater  amount  of  visual  momentum  is  said  to  help  the  operator  to  maintain  a 
cognitive  representation  of  the  system  through  presentation  of  one  display's  informa¬ 
tion  in  the  context  of  another.  The  objective  is  to  provide  "perceptual  landmarks",  or 
"anchors",  to  help  maintain  a  cognitive  representation  of  the  data.  Visual  momentum 
can  be  viewed  more  broadly  as  an  aid  to  transitioning  between  a  variety  of  points  of 
view  and  cognitive  representations. 

Visual  momentum  is  especially  important  for  military  rotorcraft  because  of  the 
frequent  need  to  transition  between  sensor  imagery,  horizontal  situation  displays,  verti¬ 
cal  situation  displays,  and  the  out-the-window  views  while  maintaining  cognizance  of 
the  spatial  relationships  of  among  battlefield  terrain,  tactical  features,  and  one's  own 
changing  position.  Although  little  experimental  work  has  been  done  to  empirically 
explore  the  characteristics  of  a  display  that  best  provide  visual  momentum,  it  is 
apparent  that  smooth  and  intuitive  transitioning  between  head-down  displays,  head-up 
displays,  and  the  real  world  should  be  one  of  the  major  goals  of  an  aircraft  display 
system 

In  searching  for  opportunities  to  use  visual  momentum  to  enhance  the  cognitive 
integration  of  information  on  display  systems,  the  first  step  is  to  examine  categories  of 
spatial  information  that  are  used  in  pilot  tasks  and  determine  whether  they  are 
currently  displayed  in  the  manner  most  natural  for  task  conduct.  To  overcome  errors 
and  incidents  in  information  transfer,  it  must  be  easy  to  integrate  new  data  with 
existing  information  regarding  the  tactical  situation  in  order  for  the  pilot  to  rapidly 
achieve  a  global  understanding  of  the  situation.  Research  in  engineering  psychology 
(e.g..  Boles  &  Wickens,  1987)  suggests  that  an  important  characteristic  that  can  facilitate 
the  integration  of  new  information  with  old  is  the  similarity  of  the  means  of  represent¬ 
ing  the  information  elements. 

For  example,  if  a  pilot's  display,  as  well  as  his  "mental  model"  of  the  information 
type  is  visual-spatial,  a  new  information  element  of  this  type  should  also  be  presented 
in  the  same  visual-spatial  manner.  This  approach  may  seem  patently  obvious,  yet  pilots 
often  need  to  mentally  transform  information  in  a  vocal  or  textual  message  to  its  visual- 
spatial  meaning  in  order  to  integrate  it  into  the  tactical  situation.  With  the  advent  of 
digital  data-link  technology,  it  is  now  possible  to  encode  the  relevant  spatial  data  in 
such  a  manner  as  to  facilitate  its  display  in  the  most  appropriate  manner,  or  in  multiple 
modalities,  locations,  and  formats,  if  the  task  so  warrants. 

The  development  of  advanced  HMD  systems  has  revolutionized  the  possibilities 
for  the  use  of  visual  momentum  as  an  aid  to  the  pilot.  As  a  result  of  these  develop¬ 
ments,  it  has  become  possible  to  ease  transitions  not  only  by  adding  spatial  symbology 
to  the  digital  map,  but  by  placing  markers  on  the  sensor  displays  or  on  the  real  world 
terrain  through  the  use  of  HMD  systems. 

The  HMD  and  Augmented  Reality.  It  is  unclear  how  far  can  the  visual  momentum 
approach  could  be  carried  in  designing  HMD  symbology.  It  is  clear,  however,  that  is 
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nearly  impossible  for  the  pilot  to  maintain  a  complete  cognitive  map  of  tactical  data 
perfectly  overlaid  on  the  visual  image  of  the  real  world.  The  HMD  provides  a  potential 
solution  to  this  serious  problem  and  its  uses  should  be  carefully  considered.  Although 
the  focus  of  HMD  development  previously  has  been  on  flight  symbology,  there  are 
scores  of  tactical  features  that  could  be  represented  in  an  "augmented  reality"  fashion 
through  the  use  of  the  HMD. 

A  "virtual  reality"  is  an  environment  or  "digital  world"  in  which  the  participant  is 
no  longer  aware  of  looking  at  a  screen,  but  becomes  "immersed"  in  the  imaginary 
scene.  "Augmented  reality,"  on  the  other  hand,  involves  methods  by  which  informa¬ 
tion  can  be  projected  on  the  real  world  to  aid  in  tasks  where  both  the  real  world  and  the 
additional  data  are  needed.  Although  the  HMD  is  the  ideal  virtual  reality  display,  its 
application  in  flight  is  that  of  an  augmented  reality  system,  overlaying  virtual  images 
on  the  real  world  (either  through  the  cockpit  monitor  or  a  sensor  display  such  as  FLIR) 
to  provide  the  required  information.  The  information  may  be  directly  related  but 
abstractly  presented,  such  as  digital  readouts  of  speed  and  altitude,  or  it  may  be 
presented  in  a  spatially  integrated  fashion  (conformal)  that  maps  onto  specific  points  in 
the  terrain,  such  as  the  horizon. 

Augmented  reality  will  be  of  enormous  aid  to  geographic  and  spatial  orientation. 
As  powerful  as  digital  map  systems  may  be,  they  are  not  capable  of  integrating  their 
information  elements  with  the  real  world  seen  out-the-window.  Instead,  the  pilot  must 
perform  several  relatively  difficult  cognitive  tasks  in  order  to  bring  the  cognitive  repre¬ 
sentation  of  head-down  symbology  into  accurate  registration  with  head-up,  real-world 
imagery. 

In  contrast,  use  of  the  HMD  permits  any  of  the  information  elements  on  the  digital 
map  to  be  presented  through  head-up  symbology,  so  that  the  positions  of  these 
elements  can  immediately  be  located  in  the  real  world.  Augmenting  reality  with  infor¬ 
mation  selected  from  the  map  display  introduces  visual  momentum  and  will  provide 
opportunities  for  integration  of  mission  management  information  in  a  more  natural  and 
useful  manner  that  has  ever  been  achieved. 

The  opportunities  thus  provided  for  the  pilot  are  extraordinary.  He  can  follow 
HMD  waypoint  symbols  along  an  axis  of  advance  while  other  symbols  warn  him  of 
areas  that  are  dangerous  due  to  contamination,  enemy  weapons,  friendly  free-fire  areas, 
or  flight  hazards.  The  locations  of  holding  areas,  phase  lines,  passage  points,  rally 
points  and  battle  positions  can  be  made  unmistakable.  Any  information  element  that 
has  an  identifiable  position  on  or  above  the  terrain  can  no  be  represented  in  that  posi¬ 
tion  in  the  real  world  instead  of  being  restricted  to  map  overlays  viewable  in  head- 
down,  plan  view  modes. 

Not  only  would  geographic  orientation  be  vastly  improved,  but  tactical  situation 
awareness  could  be  greatly  aided  by  using  the  HMD  to  present  tactical  data  such  as 
boundaries,  phase  lines,  engagement  areas,  rally  points,  and  preplanned  artillery  targets 
on  the  real  world  terrain.  Critical  new  information  elements  from  real-time  intelligence 
sources  or  combat  reports  could  also  be  depicted  in  the  terrain  as  well  as  the  map 
display.  A  preliminary  list  of  tactical  data  elements  taken  from  Army  mission 
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management  manuals  is  shown  in  the  box  below.  These  elements  are  currently  shown 
on  map  overlays,  but  could  benefit  from  presentation  by  the  HMD,  overlaying  them  on 
the  real  world  terrain  or  on  computer-generated  views  of  the  terrain  by  the  HMD 
system. 
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Figure  4-7.  Information  elements  from  map  overlays  for  augmented  reality. 

Comanche  conformal  symbols.  This  extraordinarily  powerful  augmented  reality 
capability  has  been  little  explored  in  a  tactical  setting,  although  the  Comanche  confor¬ 
mal  waypoint  symbols  have  provided  at  least  one  very  good  example  of  what  can  be 
done.  The  Comanche  helmet  integrated  display  and  sight  subsystem  (HIDSS)  was 
designed  to  provide  information  in  a  contact  analog  format,  when  possible.  One  of  the 
goals  of  the  Comanche  design  team  has  been  to  minimize  "semantic  distances".  The 
semantic  distance  is  said  to  be  large  when  the  pilot  must  think  about  his  goal  in  one  set 
of  terms  and  translate  those  thought  into  another  set  of  terms  in  order  to  decide  upon 
the  current  status.  In  spatial  terms,  minimizing  semantic  distance  is  related  to  the 
concept  of  easing  transitions  through  visual  momentum. 

Waypoint  symbols  are  "earth  stabilized"  in  the  HIDSS  system,  and  remain 
superimposed  over  their  real  world  referent  as  the  pilot  moves  his  head.  The  current 
waypoint  symbol  appears  as  a  circle  at  the  top  of  a  long  pole  in  the  earth,  as  shown  in 
Figure  4-8.  The  pole  appears  to  be  planted  in  the  earth  at  the  geographical  location  of 
the  current  waypoint.  If  the  waypoint  is  beyond  the  horizon,  the  pole  extends  only  to 
the  horizon.  The  number  in  the  circle  indicates  the  distance  to  the  waypoint.  The  next 
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waypoint  symbol  is  similar  to  the  current  waypoint  except  that  the  circle  is  smaller,  to 
give  an  impression  of  greater  distance. 


Figure  4-8.  The  Comanche  current  and  next  waypoint  symbols  (NOE-normal  mode). 

Thus,  as  the  pilot  flies  through  the  terrain,  he  is  constantly  oriented  by  the  round 
virtual  waypoints  overlaid  on  the  real  world,  which  correspond  directly  with  the  round 
waypoint  symbols  on  the  horizontal  situation  display  (on  his  digital  map).  Thus,  visual 
momentum  is  preserved  by  the  similarity  of  presentation  modality  and  formats. 

The  HMD  and  communications  improvements.  Although  the  improved  communica¬ 
tions  and  digital  map  systems  will  vastly  increase  information  sharing  between  the 
TOC  and  the  company  commander,  the  HMD  can  further  aid  these  transmissions  by 
eliminating  the  difficulties  and  errors  in  translating  between  the  real  world  and  the 
map,  and  in  position  cueing  within  and  between  aircraft. 

Communication  within  the  aircraft.  Visual  cueing  to  positions  in  the  terrain  is 
presently  rather  clumsy  in  the  AH-64,  even  though  much  improved  over  the  AH-1 
Cobra  aircraft.  The  procedure  now  used  in  the  AH-64A  to  cue  the  pilot  to  a  position  in 
the  terrain  is  as  follows: 

(1)  If  the  positions  are  preplanned  and  entered  in  the  FCC,  the  CPG  will  select 
them  with  the  Data  Entry  Keyboard  (DEK)  and  slave  the  TADS  to  them. 

(2)  To  point  out  a  position  to  the  pilot  that  is  not  pre-entered  in  the  FCC,  the 
CPG  must  enter  the  8-digit  grid  coordinates  in  the  DEK  or  find  the  position 
by  visual  search  and  fire  the  laser  to  obtain  coordinates.  He  then  slaves  the 
TADS  to  the  position. 

(3)  The  pilot  can  then  set  the  ACQ  SEL  control  to  "CPG"  so  that  the  cued  LOS 
reticle  is  the  azimuth /elevation  of  the  coordinate's  location  in  the  terrain, 
and  use  the  cued  LOS  dots  to  acquire  the  cued  LOS  reticle.  This  cue,  of 
course,  is  temporary,  lasting  only  until  the  TADS  LOS  is  redirected.  It  is  at 
this  point  that  the  pilot  is  truly  able  to  "look  where  the  CPG  is  looking." 

While  this  cueing  procedure  is  effective,  it  demands  that  the  CPG  and  his  TADS  be 
fully  dedicated  to  support  maintaining  the  pilot's  situation  awareness.  In  fact,  the 
TADS  is  primarily  in  use  for  target  acquisition  and  the  CPG  is  heavily  burdened  with 
navigation,  communication,  and  weapons  handling  tasks. 

When  a  digital  map  becomes  available  in  the  cockpit,  the  pilot's  designation  of  any 
map  position  could  result  in  the  appearance  of  a  "pointer"  symbol  identifying  that 
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position  in  the  real-world  terrain.  In  a  similar  manner,  the  pilot's  LOS  could  be  used  to 
identify  a  feature  in  the  real  world  and  enter  a  symbol  on  the  digital  map  for  immediate 
or  later  reference.  Thus,  the  difficult  problems  of  map-terrain  correlation  would  be 
overcome.  Even  in  the  near  term,  without  a  digital  map,  it  should  be  possible  for  the 
pilot  to  select  any  preplanned  position  for  "virtual  cueing,"  by  the  HMD  without 
involving  the  TADS  or  the  CPG,  as  long  as  the  head  tracker  and  navigation  system 
accuracy  are  improved. 

Communication  between  aircraft.  One  of  the  HMD's  most  important  new  capa¬ 
bilities  is  its  ability  to  point  out  terrain  or  tactical  features  to  pilots  in  nearby  aircraft. 
There  are  currently  three  methods  for  performing  target  hand-offs:  voice  with 
directions,  voice  with  coordinates,  and  laser  tracking.  For  each  method,  a  price  is  paid 
in  terms  of  performance  time,  potential  detection  by  the  enemy,  or  both. 

Target  hand-off  could  be  improved  with  a  "passive  ranging"  capability  made 
possible  through  greatly  improved  head-tracker  accuracy  and  the  incorporation  of 
digital  terrain  elevation  data  bases.  The  computation  method  is  depicted  in  Figure  4-9, 
below  and  is  performed  by  computing  the  intercept  of  the  pilot's  line  of  sight  with  the 
terrain  data  base.  The  computed  coordinates  could  then  be  sent  verbally,  or  by  "data- 
burst"  in  which  a  conformal  target  marker  is  presented  in  the  recipient's  HMD  symbol¬ 
ogy- 


Figure  4-9.  The  geometry  of  passive  ranging  with  the  HMD. 

A  passive  ranging  capability  with  a  cued  LOS  symbol  would  eliminate  the  possi¬ 
bility  of  detection  of  laser  energy  by  the  enemy,  reduce  the  risk  of  detecting  voice  traffic, 
and  speed  the  target  hand-off  process  significantly.  In  addition,  it  would  prevent  the 
inadvertent  lasing  of  friendly  troops. 

Thus,  for  attack  missions,  the  attack  team  leader  could  provide  fire  directions  by 
turning  his  head  to  direct  a  head-referenced  HMD  cursor  to  a  target  then,  by  manual  or 
voice  control,  transmit  the  target  to  a  team  member.  The  recipient's  system  would 
instantly  perform  the  geometric  computations  to  position  an  earth-referenced  HMD 
cursor  on  the  same  target.  Not  only  would  such  a  capability  be  immensely  superior  to 
current  voice  fire  control  procedures,  it  is  even  more  rapid  and  accurate  than  die 
commander  leaning  over  the  pilot's  shoulder  to  point  out  a  position  in  the  terrain 
(assuming  this  were  possible). 
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Although  target  hand-off  functions  are  the  most  obvious  application  of  the  real- 
world  pointing  capability,  any  communications  that  are  used  to  indicate  locations  of 
tactical  positions,  terrain  features,  or  other  geographic  data  can  be  greatly  aided  by  this 
HMD  display  technique. 

Communication  with  the  TOC.  The  same  kind  of  technique  is  also  valuable  for 
transmitting  report  information  to  the  TOC  or  ground  unit  commander.  Currently,  if  a 
pilot  sees  a  potential  target  in  the  real  world  and  wants  to  determine  the  precise  loca¬ 
tion  of  the  target  site  without  risking  exposure  by  lasing,  he  must  determine  his  present 
location  from  the  navigation  computer,  estimate  range  and  bearing  to  the  target  from 
his  own  position,  and  attempt  to  determine  the  UTM  coordinates  on  a  paper  map. 

These  6-  or  eight-digit  grid  coordinates  are  then  transmitted  to  the  recipient  for  plotting 
on  another  map. 

With  the  HMD  (and  a  digital  terrain  database),  the  pilot  would  simply  look  at  the 
target,  bringing  a  location-marking  cursor  over  the  target  position.  The  computer 
system  would  then  calculate  the  HMD  line-of-sight  and  terrain  grid  coordinates.  Such  a 
system  could  be  used  to  instantly  place  a  symbol  on  a  digital  map  display  correspond¬ 
ing  to  the  computed  target  location,  and  at  the  pilot's  command,  to  transmit  this  posi¬ 
tion  to  the  TOC.  The  TOC  could,  in  return,  data-link  important  position,  line,  or  area 
data  to  the  unit  commander  for  optional  display  via  the  commander's  HMD,  overlaid 
on  the  terrain.  Importantly,  problems  of  mistaken  landforms  or  geographic  coordinate 
identification  errors  are  not  introduced  in  either  process. 

Masking  and  Intervisibility  Issues.  The  HMD  can  present  specific  line-of-sight,  cover, 
and  concealment  information  in  the  views  and  formats  most  compatible  with  the  pilot's 
requirements  when  evaluating  masking,  fields  of  fire,  backdrop,  and  related  questions. 

An  HMD  application  in  this  area  is  the  enhancement  of  various  line-of-sight 
computations  by  presenting  the  three-dimensional  forms  that  are  not  readily  presentable 
with  a  top-down  view.  For  example,  a  difficulty  with  depicting  threat  weapon 
effectiveness  envelope  on  the  digital  map  alone  is  that  the  top-view  cannot  show  the 
potential  changes  in  the  envelope,  based  upon  differences  in  altitude  above  the  ground. 
With  the  head-up  view  offered  by  the  HMD,  the  vertical  profile  of  the  threat  weapon 
envelopes  could  be  depicted  by  such  symbology,  as  shown  in  Figure  4-10. 

In  addition,  the  HMD  could  use  symbols  to  show  the  pilot  the  location  of  any 
given  point  (such  as  a  waypoint  or  an  enemy  weapon)  and  use  some  visual  code  to 
indicate  whether  the  position  is  actually  visible  from  the  current  location,  or  is  posi¬ 
tioned  behind  the  currently  visible  terrain.  Such  a  capability  will  be  described  in  more 
detail  in  Section  6  of  this  report. 

Furthermore,  as  new  enemy  situation  information  is  transmitted  to  the  aircraft,  the 
HMD  can  also  be  employed  to  warn  the  pilot  of  his  potential  visibility  to  enemy 
weapons,  and  suggest  routes  to  the  closest  cover  and  concealment. 
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Figure  4-10.  A  fanciful  example  of  an  HMD  3-D  threat  representation  suggested  for  the 
Advanced  Tactical  Fighter  (Northrop/McDonnell  Douglas,  1988). 


Mission  Information  Requirements  Analysis  and  HMD  Symbology 

In  order  to  examine  the  variety  of  new  HMD  symbols  potentially  valuable  to  the 
pilot,  it  is  first  necessary  to  analyze  the  required  information  content  of  the  symbols. 

The  most  effective  way  to  proceed  is  to  extend  the  traditional  task  analysis  to  create  a 
cognitive  task  analysis  approach.  To  extend  the  list  of  candidate  symbols,  intensive 
interviews  with  expert  pilots  and  surveys  of  symbol  importance  must  also  be 
conducted.  In  addition,  in  order  to  accommodate  future  information  requirements, 
careful  examination  of  MEP  additions  must  be  undertaken  for  each  new  project. 

Identify  Emerging  HMD  Mission  Information  Presentation  Opportunities 

During  the  Phase  I  effort,  we  had  selected  a  set  of  in-flight  activities  to  create  a 
representative  range  of  information  elements  and  pilot  information  processing 
functions.  The  activities  were  specifically  selected  to  include  events  associated  with 
changing  symbology  needs.  In  order  to  ensure  that  supporting  SMEs  would  be  avail¬ 
able,  current  AH-64  tasks  served  as  the  pool  from  which  the  to-be-analyzed  activities 
were  drawn.  For  the  Phase  II  effort,  we  put  additional  emphasis  on  the  types  of  mission 
functions  that  will  be  representative  of  those  to  be  conducted  by  aviators  in  the  foresee¬ 
able  future,  in  more  advanced  aircraft  and  using  more  powerful  avionics  capabilities,  so 
that  our  findings  would  be  applicable  over  the  long-term. 
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In  particular,  we  believe  that  HMD  systems  hold  promise  for  solving  problems  in 
the  areas  of  tactical  communications,  terrain  interpretation,  masking  and  intervisibility 
determination,  geographic  orientation,  and  situation  awareness.  We  attempted  to 
identify  the  additional  pilot  tasks  and  related  information  elements,  and  specify  ways 
that  the  required  HMD  capabilities  would  change  as  a  result  of  these  improvements. 
Some  examples  of  these  changing  tasks  and  technologies  are  described  in  this  section. 

Historically,  most  task  analyses  have  been  based  largely  on  the  control  inputs  and 
display  outputs  of  selected  MEP  components.  In  failing  to  examine  the  specific  appli¬ 
cations  of  these  components,  the  simultaneous  task  requirements  of  combat  and  the 
resulting  workload  thrust  upon  the  pilot,  such  analyses  do  not  always  reveal  the  infor¬ 
mation  elements  that  most  directly  support  the  pilot  in  meeting  special  operational 
problems. 

To  have  an  in-depth  understanding  of  the  current  problems  in  acquiring,  inter¬ 
preting,  and  using  battlefield  data  in  the  cockpit,  it  is  critical  to  determine  the  impact  of 
the  information  requirements  within  the  context  of  mission  management  functions, 
examine  the  current  sources  of  the  necessary  information  and,  by  inference,  the  defi¬ 
ciencies  experienced  in  an  on-board  setting.  Based  on  such  analyses,  and  upon  exten¬ 
sive  interviews  with  SMEs  from  operational  units,  it  is  possible  to  identify  the  salient 
information  problems  posed  by  current  systems  and  suggest  feasible  contributions  to  be 
made  by  improved  HMD  systems. 

Source  of  the  Task  Analysis  and  Workload  Information 

For  many  years  Anacapa  Sciences  has  performed  task  analysis  and  workload 
studies  for  the  Army,  employing  teams  of  psychologists  and  helicopter  pilots  to  capture 
analytic  data  useful  in  designing  and  evaluating  new  aircraft  and  avionics  concepts. 
Several  years  ago  (Hamilton,  Bierbaum,  &  Rogers,  1992)  we  began  a  series  of  analyses  to 
extend  the  existing  data  to  address  HMD  issues.  The  existing  databases  identifying 
AH-64  Apache  mission  phases,  segments,  functions,  and  tasks  were  further  decom¬ 
posed  into  lists  of  individual  information  elements  required  to  perform  the  tasks  in  a 
scout-attack  mission,  and  the  specific  attributes  of  each  information  element  were 
categorized. 

The  analyses  were  iteratively  reviewed  by  Army  pilots  and  psychologists  and 
organized  into  a  relational  database  called  the  Task  Analysis  and  Work  Load  Operator 
Relational  Database  for  Information  Requirements,  or  TAWL  ORDIR.  By  the  conclu¬ 
sion  of  the  effort  (Hamilton,  Rogers,  &  Spiker,  1995),  some  9  phases,  86  segments,  234 
functions,  554  tasks,  567  requirements,  and  667  information  elements  had  been  identi¬ 
fied  for  the  conduct  of  attack  and  reconnaissance  missions.  The  mission  analysis 
document  uses  a  hierarchical,  semi-chronological  approach  to  putting  the  information 
elements  in  a  meaningful  context  and  is  presented  in  a  two-volume,  1200-page  report. 

TAWL  ORDIR  System  Design  and  Organization 

TAWL  ORDIR  was  developed  by  Anacapa  Sciences  to  perform  human  factors 
analyses  of  complex  system  requirements  (Hamilton,  Rogers,  &  Coker,  1994).  This 
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relational  database  was  designed  to  dynamically  store,  organize,  and  display  the 
enormous  amount  of  information  that  is  inventoried  during  analyses  of  complex 
systems,  increasing  analyst  productivity  and  enhancing  the  utility  of  the  resulting  ana¬ 
lytical  research  data.  The  software  stores  the  data  in  a  manner  that  facilitates  the  devel¬ 
opment,  maintenance,  and  documentation  of  the  analysis  using  a  graphical  user  inter¬ 
face  and  a  series  of  on-line  and  printed  reports. 

The  basic  structure  of  TAWL  ORDIR  databases,  shown  in  Figure  4-11,  supports  the 
conduct  of  top-down  mission  and  information  analyses.  Mission  analysis  components 
are  identified  at  the  six  levels  presented  in  the  boxes  on  the  left  of  the  figure:  segments, 
functions,  tasks,  requirements,  elements,  and  sources.  Each  of  the  components  identi¬ 
fied  during  the  mission  analysis  is  stored  in  the  system  as  a  simple  list  or  "flat  file."  The 
items  in  each  of  the  lists  are  then  related  to  items  in  the  level  of  analysis  just  above  and 
below  them.  Thus,  to  define  the  tasks  in  a  function,  data  are  stored  that  relate  that 
function  to  its  constituent  tasks.  Similarly,  functions  are  related  to  the  segments  in 
which  they  occur.  These  relations  are  represented  in  the  figure  using  arrows  to  link  the 
levels  of  the  mission  analysis. 
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Figure  4-11.  The  organization  of  TAWL  ORDIR  databases. 

The  information  analyses  include  information  attribute  ratings  for  the  nine  infor¬ 
mation  attributes  presented  in  boxes  on  the  right-hand  portion  of  the  figure.  Attribute 
ratings  are  made  at  different  levels  of  the  mission  analysis.  As  can  be  seen  from  the 
figure,  one  attribute  is  rated  at  the  function  level  (Information  Category);  three  attrib¬ 
utes  are  rated  at  the  task  level  (Cognitive,  Perceptual,  and  Psychomotor  Nature),  and 
five  attributes  are  rated  at  the  element  level  (Spatial  Reference,  Data  Level,  Temporal 
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Nature,  Transience,  and  Tactical  Nature).  Attribute  ratings  are  defined  in  the  database 
by  storing  data  that  relate  the  mission  components  to  their  constituent  information 
attribute  ratings.  These  relations  are  represented  in  the  figure  using  arrows  to  link  the 
information  attributes  with  the  components  of  the  mission  analysis. 

For  HMD  symbology  designers,  the  most  important  level  of  analysis  is  that  of 
identifying  the  information  elements  that  are  needed  to  meet  the  pilot's  information 
requirements.  Information  elements  are  the  units  that  must  be  monitored,  interpreted, 
transformed,  and  integrated  by  the  pilot  to  view  the  desired  and  actual  state  of  the 
system  and  to  answer  the  questions  posed  by  the  information  requirements.  For  exam¬ 
ple,  the  answer  to  the  question  "Is  stable  flight  being  maintained?"  is  provided  by  the 
information  elements  "Vertical  clearance  of  obstacles,"  "Horizontal  clearance  of  obsta¬ 
cles,"  "Altitude  above  ground  level,"  "Airspeed,"  "Ground  speed,"  "Pitch,"  "Roll," 
"Drift  direction,"  and  "Heading  to  feature." 

This  portion  of  the  analysis  was  especially  important  because  the  information 
elements  identified  during  this  process  define  what  must  be  available  to  the  pilot,  either 
from  the  mission  equipment  and  aircraft  displays  or  from  the  pilot's  perceptual  analysis 
of  the  available  sensor  images  or  out-the-window  views.  Importantly,  the  resulting  list 
of  information  elements  forms  the  baseline  for  evaluating  a  display  system  and  helps  to 
identify  deficiencies,  duplications,  conflicts,  or  other  shortcomings  in  the  content  of  the 
information  presented,  as  well  as  provide  clues  to  the  development  of  new  and  superior 
display  systems. 

We  have  found  TAWL  ORDIR  to  be  particularly  useful  in  considering  intelligent 
symbology  management  because  of  its  ability  to  immediately  determine  and  display  the 
information  elements  used  in  any  segment,  function,  or  task,  or  the  specific  segments, 
functions,  and  tasks  that  require  a  given  information  element.  In  addition  to  aiding  in 
these  analyses,  TAWL  ORDIR  could  also  be  used  for  rapid  retrieval  of  information 
elements  for  discussion  during  the  SME  interviews. 

For  example,  as  shown  in  Figure  4-12,  TAWL  ORDIR  allows  the  investigator  to 
select  one  or  more  information  elements  from  the  list  of  elements  and  to  request  lists  of 
the  mission  components  (such  as  segments,  functions,  and  tasks)  in  which  those 
elements  are  required.  Alternatively,  the  investigator  can  display  the  mission  compo¬ 
nents  in  which  those  elements  are  not  required.  This  type  of  procedure  is  useful  for 
determining  the  relative  frequency  of  use  and  the  criticality  of  selected  information 
elements. 

In  addition,  we  used  these  procedures  to  select  one  or  more  functions  from  the  list, 
and  then  generate  a  list  of  the  information  elements  required  to  perform  the  selected 
functions,  as  shown  in  Figure  4-13.  Applying  this  approach,  we  could  instantly  define 
the  apparent  set  of  information  requirements  for  display  modes  tailored  to  support  a 
particular  set  of  mission  functions.  Carrying  the  database  on  a  laptop  computer,  we 
were  able  to  review  those  information  elements  with  SMEs  at  both  Fort  Rucker  and 
West  Jordan  by  constructing,  "on-the-fly,"  sets  of  necessary  elements  to  serve  in  various 
tasks  or  functions. 
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Figure  4-12.  The  information  element  view  in  Figure  4-13.  The  mission  components 
TAWL  ORDIR.  view  in  TAWL  ORDIR. 


The  TAWL  ORDIR  Mission  Analysis  Report 

TAWL  ORDIR  was  designed  to  be  flexible  both  in  the  structure  of  the  information 
stored  in  its  databases  and  in  its  functional  capabilities.  The  design  specifications  have 
emphasized  the  storage,  display,  and  documentation  of  the  information  gathered 
during  task  and  information  analyses.  Several  types  of  TAWL  ORDIR  on-line  and 
printed  reports  have  been  developed.  The  easiest  to  understand,  however,  is  the 
Mission  Analysis  Report  because  of  its  hierarchical  organization.  It  is  similar  to  tradi¬ 
tional  task-analytic  techniques  in  its  presentation  of  mission  analysis  data  in  a  semi- 
chronological  fashion. 

For  example,  each  line  on  the  screen  shown  in  Figure  4-14  presents  one  labeled 
component  of  the  mission  analysis.  The  first  line  of  the  report,  labeled  "segment,"  gives 
the  segment  name  (NOE  Flight)  and  shows  the  segment  identifier  in  parentheses  (1). 

The  second  line  introduces  the  first  function,  gives  its  name  (Retrieve  Saved  Message), 
and  shows  the  function  identifier  (105).  The  third  line  introduces  the  first  task,  gives  the 
task  name,  and  shows  the  task  identifier.  Similarly,  one  requirement  with  two  infor¬ 
mation  elements  and  their  associated  sources  are  given  to  complete  the  description  for 
the  first  task  in  the  function.  The  other  tasks  in  the  function  are  then  described  in 
sequence.  The  order  of  the  tasks  in  the  functions  is  consistent  with  the  order  in  which 
they  are  expected  to  be  performed  during  the  mission.  Functions  are  repeated  under 
each  segment  in  which  they  occur. 


iSegmenf 


NOE  Flight  (1) 


Function  ->  Retrieve  Saved  Message  (105) 

Task  -»  Enable/Disable  Message  Retrieval  (250) _ 

Requirement  _>  Has  message  been  retrieved?  (228) 

Element  ->  Time  of  Arriva|  (93) 

source  -h>  Digital  Message  Content  (28) 
Bement  Time  0f  Activity  (95) 

source  0jgjta|  Message  Content  (28) 


Figure  4-14.  A  portion  of  the  Mission  Analysis  Report  for  one  segment  of  the  mission. 
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The  completed  mission  analysis  document  offered  a  hierarchical,  semi-chronologi¬ 
cal  approach  to  putting  the  information  elements  in  a  meaningful  context  and,  when 
printed  out,  resulted  in  a  two-volume,  1200-page  report. 

Identifying  Potential  HMD  Applications 

To  identify  the  individual  information  elements  that  could  beneficially  be 
presented  on  the  HMD,  we  employed  the  TAWL  ORDIR  information  requirements  data 
as  presented  in  the  1200-page  Mission  Analysis  Report.  The  objective  was  to  pass  the 
information  requirements  data  through  a  series  of  logical  "filters"  to  identify  tasks  for 
which  the  information  elements  might  best  be  presented  on  the  HMD.  A  psychologist 
and  an  Apache  subject  matter  expert  painstakingly  combed  through  the  report  in 
attempting  to  systematically  reduce  the  list  of  elements  to  only  those  that  might  be 
found  on  a  future  HMD  system  with  intelligent  moding. 

The  first  filter  was  the  selection  of  mission  segments  and  tasks  that  clearly  involve 
the  use  of  the  HMD,  including  all  flight  and  tactical  engagement  components.  TAWL 
ORDIR  sorts  were  performed  to  identify  all  elements  required  to  perform  every 
applicable  mission  segment  and  function. 

For  example,  a  sort  of  "Segment  14,  Landing,"  results  in  a  list  of  information 
elements  such  as  "Altitude  AGL,"  "Pitch,"  "Roll,"  "Torque,"  and  some  less  expected 
elements  such  as  "Status-tailwheel"  and  "Looming  objects."  To  further  explore  the 
nature  of  these  elements,  TAWL  ORDIR  was  used  to  observe  them  in  the  context  of  a 
Mission  Analysis  print-out,  showing  the  hierarchical  relations  of  mission  segments, 
functions,  tasks,  and  elements.  Figure  4-15  shows  a  portion  of  the  landing  sequence 
from  a  TAWL  ORDIR  Mission  Analysis  printout.  The  analysis  shows  tasks,  information 
requirements,  information  elements,  and  information  sources  in  context. 

Additional  information  analysis  options  from  TAWL  ORDIR  include  Mission 
Timeline  Reports,  which  orders  the  pilot's  functions  by  when  they  are  performed  in 
time,  as  well  as  the  predicted  workload  in  cognitive,  visual,  auditory,  and  psychomotor 
categories  as  they  are  associated  with  the  moment-to-moment  pilot  activities.  High 
workload  prediction  ratings  are  particularly  useful  in  identifying  opportunities  for 
automatic  intelligent  symbology  management  of  individual  information  elements. 

The  selection  of  the  subsequent  series  of  filters  was  based  on  the  established  bene¬ 
fits  of  HMD  systems.  First,  a  primary  advantage  of  the  HMD  is  simply  that  it  mini¬ 
mizes  head-down  time,  a  particularly  valuable  asset  for  terrain  flight  in  military 
helicopters.  Thus,  tasks  most  requiring  that  the  pilot  scan  the  surrounding  terrain  and 
airspace  were  closely  inspected  for  opportunities  to  intelligently  present  required 
information  elements. 

Another  filter  addressed  the  ability  of  the  HMD  to  present  conformal  symbology 
in  which  the  symbol  element  that  represents  a  real-world  feature  actually  marks  the 
feature's  location  in  the  pilot's  field  of  view,  staying  on  that  marked  position  even  as  the 
pilot  moves  his  head.  As  Weintraub  (1994)  has  noted,  "Conformality  is  a  critical 
advantage  that  cannot  be  duplicated  head  down  if  the  eye  must  also  gather  information 
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Transit  to  Forward  Rendezvous  (4) _ 

Awcrtoff  — *  Land  Aircraft  (26) 

Tm*  ->  Control  Attitude  (4} _ _ 

x^^*™**  ts  the  aircraft  nose  too  higft?  (4) 

Cfcnm -»  Pitch  (5) 

Siwo*  -+  Heading  and  Attitude  Reference  System  (9} 

5o iwi  -►  Visual  Scene  {sensor)  (28) 

**&*«*«* r  -»  is  the  aircraft  nose  too  tow?  (5) 

£W«»fir  -»  pitch  (5) 

Heading  and  Altitude  Raioronco  System  {9) 
so***  -»  Visual  Scene  (senstx)  (28) 

is  a  wing  too  tow?  (6) 

SSfnwtf  -*  Roll  (17) 

saws*  -*  Heading  and  Attitude  Reference  System  (9) 
sows*  -*  Visual  Scene  (sensor)  (28) 

Ak^tv^mmr  -*  vviiat  Is  the  U ight  director's  pitch  command?  (532) 

&p/n**,r  Pttch  Command  -  Flight  Director  (529) 

Sain*  Automatic  Flight  Control  Systom  (22) 

-»  what  is  the  flight  director's  roll  command?  (533) 

&orr>**i  Bol  Command  -  Right  Director  (522) 

-*  Automatic  Flight  Control  System  (22) 

w  ^  Maintain  Obstacle  Clearance  (32) _ _ 

********  ->  is  the  aircraft  high  enough  to  dear  obstacles?  (2) 

s*vn*nr  Distance  Vertical  from  Obstacles  (454) 
savre*  ->  Visual  Scene  (sensor)  (28) 

****** frwni  -*  Turn  /  move  to  avoid  obstacles?  (7) 

»  Distance  Horizontal  from  Obstacles  (18) 

-*  Visual  Scene  (sensor)  (28) 

7**  -+  Control  Drift  (36) _ 

-*  vVhat  Is  the  magnitude  and  diredton  of  aircraft  movement?  (41) 
©««nr  Speed  Ground  (8) 

fcufw  Navigation  System  (15) 
soiiw  Visual  Scene  (sensor)  (28) 

Drift  Direction  (83) 
s<ww  -►  Navigation  System  (15) 
swvw  ■<-*  Visual  Scene  (sensor)  (28) 

r«*  -4  Control  Heading  for  Stable  Flight  (50) _ ___ 

xtqutomm*  -*  is  tho  aircraft  heading  steady  or  changing  at  a  steady  rate?  (52) 
Sffm*nr  -+  Magnetic  Heading -Ownshlp  (18) 

Heading  and  Attitude  Reference  System  (9) 
£km**  — >  Heading  to  Feature  (84) 


Figure  4-15.  Example  from  TAWL  ORDIR  Mission  Analysis  printout. 
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directly  from  the  visual  environment."  Thus,  we  studied  the  task-analytic  data  to  iden¬ 
tify  the  pilot  functions  involving  specific  cartographic  or  terrain  positions  that  could  be 
intelligently  presented  by  "augmenting"  reality  (i.e.,  superimposing  symbology  on  real- 
world  locations  as  viewed  through  the  HMD). 

Yet  another  filter  was  related  to  the  HMD's  very  direct  method  of  "pointing"  at 
features  in  the  environment  and  cueing  the  CPG,  or  other  HMD-equipped  aviators  in 
other  friendly  aircraft,  to  look  at  the  position  pointed  out.  Tasks  involving  these  types 
of  elements  were  scrutinized  for  intelligent  display  opportunities. 

In  short,  the  objective  was  to  search  the  mission  data,  item-by-item,  for  cases  in 
which  the  established  criteria  for  HMD  use  were  applied  to  the  functions,  tasks,  and 
information  requirements  described  in  the  task  analysis  data.  These  cases  were 
recorded  for  use  during  the  subsequent  SME  interviews  and  survey  so  that  the  apparent 
"hits"  could  be  confirmed. 

To  maintain  their  focus  on  the  objective  of  the  analysis  and  its  logical  filters,  the 
review  team  members  were  guided  by  eleven  ground  rules  for  including  elements  or 
excluding  elements,  as  described  below. 

First,  it  would  almost  always  be  important  to  include  elements  that  are: 

1.  Required  for  tasks  that  clearly  involve  HMD  use,  such  as  gun  aiming. 

2.  Used  during  tasks  that  demand  head-up  viewing,  such  as  flight  control 

3.  Potentially  located  by  HMD  pointing  and  cueing  symbols  (e.g.  wires) 

4.  Refer  to  spatial  positions,  lines,  or  areas  on  or  above  the  earth. 

Second,  it  would  almost  always  be  important  to  exclude  elements  that  are: 

1.  Used  only  in  the  Mission  Preparation  phase,  before  the  HMD  is  donned. 

2.  Part  of  the  real-world  scene,  such  as  trees  or  rivers  (direct  view  or  sensor) 

3.  Necessarily  in  the  form  of  extensive  text  like  checklists  and  orders. 

4.  System  status  information  such  as  intercom  mode,  or  chaff  dispenser  mode 

5.  Current  system  configuration  such  as  exterior  lighting  and  de-icing 

6.  Specific  system  advisories  (beyond  a  master  warning  or  caution  indicator) 

7.  Voice  communications  (unless  the  data  might  better  be  presented  on  the  HMD. 

Although  tedious,  it  was  more  useful  to  go  through  the  1200  pages  of  mission 
analyses  than  to  simply  examine  a  list  of  the  667  information  elements  because  the 
mission  analysis  report  presented  the  information  elements  in  the  context  of  mission 
functions  and  tasks.  Simply  examining  a  list  of  the  667  information  elements  would  not 
always  have  made  evident  the  value  of  an  HMD  symbol.  Furthermore,  when  examined 
in  context,  the  same  element  might  be  included  or  excluded,  depending  upon  the 
differing  nature  of  the  mission  tasks. 

Results  of  the  Review 

The  results  of  the  review  were  that  124  information  elements  were  identified  as 
potentially  useful  on  an  HMD  display.  The  elements  are  listed  below,  along  with  their 
identifier  numbers  in  the  TAWL  ORDIR  data  base.  The  information  elements  have  been 
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grouped  into  five  categories:  aviation  &  navigation,  threat  &  weapons  management, 
zones  and  positions  on  earth,  positions  in  image,  zones  and  positions  in  space. 


Aviation  &  Navigation 

2.  Altitude  -  above  ground  level 

3.  Speed  air 

4.  Yaw 

5.  Pitch 

6.  Speed  ground 

16.  Distance  horizontal  from  obstacles 

17.  Roll 

20.  Altitude  -  relative  to  surrounding  terrain 
43.  Torque  -  total 
79.  Wires 
81.  Towers 

83.  Drift  direction 

84.  Heading  to  feature 
88.  Target  bearing 

106.  Torque  -  engine  #1 

107.  Torque  -  engine  #2 


116.  Altitude  -  mean  sea  level 

117.  MSL  altitude  -  planned 

124.  Speed  vertical 

125.  Local  terrain  topology 
238.  Speed  ground  -  planned 

305.  Speed  air  -  minimum  single  engine 
375.  Time  before  damage  by  high  torque  condition 
483.  Distance  -  object  of  interest 
454.  Distance  vertical  from  obstacles 

501.  Altitudes  -  minimum  obstruction  clearance 

502.  Altitudes  -  minimum  enroute 

527.  AGL  altitude  -  planned 

528.  Roll  command  -  flight  director 

529.  Pitch  command  -  flight  director 

665.  Looming  objects 

666.  Objects  of  interest 


Threat  &  Weapons  Management 

7.  Distance  to  most  dangerous  threat 

8.  Bearing  to  most  dangerous  threat 

9.  Threat  searching 

10.  Threat  lock-on 

11.  Missile  launch  -  threat 

13.  Missile  constraints 

14.  Status  -  weapon  launch 

15.  Missile  separation 

18.  Magnetic  heading  -  ownship 

19.  Magnetic  heading  -  planned 
21.  Threat  intervisibility 

Zones  and  Positions  on  Earth 

25.  Position  on  earth  -  marked 
70.  Positions  on  earth  -  intersections 
137.  Positions  on  earth  -  ground  targets 

144.  Positions  on  earth  -  friendly  artillery 

145.  Positions  on  earth  -  active  artillery 
148.  Zone  on  earth  -  assigned  attack 
150.  Zones  on  earth  -  friendly  locations 
159.  Zones  on  earth  -  enemy  locations 
161.  Zone  on  earth  -  fire 

163.  Zones  on  earth  -  no  fire 

165.  Positions  on  earth  -  threat  locations 

174.  Remote  location's  view  of  battlefield 

175.  Remote  location's  cover 

176.  Remote  location's  concealment 
179.  Positions  on  earth  -  ground  targets 

detected  by  radar 

154.  Position  on  earth  -  checkpoints 
262.  Zone  on  earth  -  kill 


23.  Selected  weapon 
26.  Rate  -  gun  fire 
36.  Enemy  fire 

60.  Time  of  guided  missile  flight 
173.  Altitude  -  target  array 

177.  Guided  missile  launch  -  ownship 
348.  Code  -  range  finder 
396.  Status  -  fire  and  forget  missile  tracker 
402.  Time  to  designate 
443.  Status  -  air  to  air  missile  tracker 


394.  Overlay  -  air  strike  missions 

397.  Overlay  -  flight  hazards 

412.  Positions  on  earth  -  selected  ground  targets 

437.  Zone  on  earth  -  target  area 

485.  Position  on  earth  -  selected  air  strike  mission  target 

486.  Position  on  earth  -  selected  artillery  mission  target 
506.  Zone  on  earth  -  weapon  impact  area 

538.  Positions  on  earth  -  ownship's  projection 

544.  Positions  on  earth  -  navaids 

545.  Positions  on  earth  -  airports 

600.  Positions  on  earth  -  force  boundaries 
609.  Zones  on  earth  -  engagement  areas 

620.  Positions  on  earth  -  holding  areas 

621.  Positions  on  earth  -  phase  fines 
636.  Positions  on  earth  -  FARP  locations 

641.  Positions  on  earth  -  downed  aviator  pickup 
points 

652.  Positions  on  earth  -  assembly  areas 
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Positions  in  Image 

29.  Position  in  image  -  helmet  pointer 
35.  Position  in  image  -  ownship  location 
on  map 

37.  Positions  in  image  -  targets  detected 
unaided 

Zones  and  Positions  in  Space 

32.  Zones  in  space  -  threat  lethality 
41.  Positions  in  space  -  actual  flight  route 
89.  Position  in  space  -  next  to  shoot  target 
100.  Position  in  space  -  refueling  tanker 
147.  Positions  in  space  -  flight  team  members 

152.  Position  in  space  -  selected  waypoint 

153.  Positions  in  space  -  selected  flight  route 

156.  Positions  in  space  -  active  flight  route 

157.  Positions  in  space  -  waypoints 
160.  Zones  in  space  -  restricted  areas 
162.  Position  in  space  -  active  waypoint 
172.  Position  in  space  -  rendezvous  aircraft 
392.  Position  in  space  -  home  waypoint 
482.  Position  in  space  -  helmet  boresight 
530.  Positions  in  space  -  air  targets 


Positions  in  space  -  air  targets  detected  by  radar 
Position  in  space  -  refuel  location 
Position  in  space  -  ownship  present  position 
Positions  in  space  -  initial  approach  fixes 
Positions  in  space  -  step  down  fixes 
Positions  in  space  -  instrument  approach 
Positions  in  space  -  missed  approach 
Position  in  space  -  final  approach  fix 
Position  in  space  -  missed  approach  point 
Positions  in  space  -  approach  holding  patterns 
Zones  in  space  -  air  routes  and  corridors 
Positions  in  space  -  air  control  points 
Positions  in  space  -  rally  points 
Positions  in  space  -  battle  positions 
Zones  in  space  -  NBC  contamination 


61.  Positions  in  image  -  targets  detected  sensor 

390.  Position  in  image  -  air  to  air  missile  tracker 

391.  Position  in  image  -  fire  and  forget  missile 
tracker 

508.  Position  in  image  -  object  of  interest 


531. 

536. 

537. 
552. 
554. 

561. 

562. 

567. 

568. 

569. 

617. 

618. 
619. 
622. 
651. 


Mission  Equipment  Package  Implications  for  HMD  Symbology 

The  preceding  pages  have  described  the  operational  requirements  "pull"  on  HMD 
symbology  design  through  consideration  of  the  mission  task  information  that  must  be 
processed  by  pilots  of  scout-attack  rotorcraft.  It  is  important  to  remember,  however, 
that  there  are  HMD  symbology  design  issues  stemming  from  the  technology  ’’push"  of 
ever-increasing  avionics  capabilities.  The  particular  mission  equipment  package  (MEP) 
installed  in  the  aircraft  serves  to  constrain  and  shape  the  type,  amount,  and  flow  of 
information  to  the  pilot.  Advanced  avionics  technologies  will  greatly  change  the  infor¬ 
mation  available  to  the  pilot  in  the  near  term.  For  example,  the  Rotorcraft  Pilot's  Asso¬ 
ciate  (RPA)  may  provide  "cognitive  decisions  aids"  that  recommend  specific  pilot 
actions;  the  Obstacle  Avoidance  System  (OASYS)  or  the  Terrain  and  Obstacle  Detection 
Sensor  (TODS)  may  provide  advance  warning  of  wires  and  towers  in  the  flight  path;  the 
Integrated  Air-to-Air  Weapon  (INTAAW)  may  improve  the  short-range  effectiveness  of 
an  automatic  cannon;  Tactics  Planning  Expert  Functions  may  generate  flight  routes 
based  on  the  threat  situation;  advanced  Data  Transfer  Systems  may  permit  storage  and 
transmission  of  massive  amounts  of  mission  planning  data;  and  advanced  ASE  may 
present  integrated  lethality  data. 

Traditional  information  requirements,  such  as  "where  is  the  enemy?"  have  not 
changed  over  the  years.  However,  as  the  above  sample  of  new  technologies  shows, 
emerging  systems  will  address  these  traditional  requirements  in  dramatically  different 
ways.  Less  obvious  are  the  impacts  these  technologies  will  have  on  HMD  symbology. 
The  particular  mission  equipment  package  (MEP)  installed  in  the  aircraft  serves  to 
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constrain  and  shape  the  type,  amount,  and  flow  of  information  to  the  pilot.  Since  the 
HMD  is  the  pilot's  window  not  only  to  his  flight  and  tactical  environment,  but  also  to 
the  new  information  available  from  the  MEP  operational  and  technical  capabilities, 
effectively  presenting  the  new  symbology  without  cluttering  the  field  of  view  is  critical, 
and  success  will  almost  certainly  be  dependent  upon  developing  an  intelligent  symbol¬ 
ogy  management  system. 

Avionics  Context  of  the  HMD 

Needless  to  say,  effective  HMD  symbology  design  cannot  proceed  without  consid¬ 
ering  the  avionics  context  in  which  the  HMD  itself  will  operate.  That  the  aircraft  will 
have  an  HMD  implies  that  some  decisions  about  its  associated  MEP  configuration  (e.g., 
using  an  HMD  in  lieu  of  a  HUD,  making  extensive  use  of  advanced  night  sensors)  will 
likely  have  been  made.  But  once  the  HMD  has  been  included  in  the  MEP,  more  fine¬ 
grained  decisions  concerning  the  selection  and  design  of  its  symbology  must  inevitably 
depend  on  the  specific  avionics  systems  that  will  serve  as  the  source  for  the  various  to- 
be-displayed  information  elements. 

As  the  name  suggests,  an  MEP  is  configured  to  accomplish  the  specific  missions 
assigned  a  given  aircraft.  Our  focus  has  been  on  scout-attack  weapon  systems,  which  in 
comparison  to  their  fixed  wing  counterparts,  are  highly  weight-constrained.  As  such, 
many  avionics,  while  technologically  quite  feasible,  simply  cannot  be  put  into  the  cock¬ 
pit.  Even  compared  to  heavier  utility  and  lift  helicopters,  scout-attack  avionics  are  quite 
limited.  A  case  in  point  is  the  Air  Force’s  Special  Operations  Forces  (SOF)  MH-53J  Pave 
Low  III  heavy  lift  helicopter.  The  Pave  Low  contains  a  number  of  advanced  (but  heavy) 
technologies  not  found  in  Army  scout-attack  helicopters.  These  include  terrain 
following /terrain  avoidance  radar,  advanced  electronic  countermeasures  (ECM)  and 
jamming,  hover  coupler  control,  and  integrated  defense  avionics  (Spiker,  1995). 

The  impact  of  size  and  weight  constraints  on  scout-attack  avionics  selection  is  an 
important  consideration,  as  it  determines  the  technology  "filter"  through  which  the 
symbology  must  pass.  In  this  regard,  refueling  tactical  aircraft  during  a  mission  is  espe¬ 
cially  risky,  whether  from  the  air  or  the  ground.  Air-to-air  refueling  requires  that 
aircraft  ascend  to  higher  tanker  altitudes,  exposing  all  to  enemy  groundfire.  Landing  at 
a  ground-positioned  FARP  is  equally  dangerous,  as  complex  logistics  are  required  to 
ensure  safe  ingress  and  egress.  Consequently,  there  is  a  tactical  requirement  to  save  as 
much  weight  as  possible  for  fuel,  in  order  to  increase  the  aircraft's  operational  range. 

During  Desert  Storm,  for  example,  both  two  battalions  of  AH-64As  deployed  in 
Iraq  were  outfitted  with  the  extended  range  fuel  system  (ERFS).  The  ERFS  contains 
auxiliary  fuel  tanks  that  permit  the  heavy  Apaches  to  fly  700nm  without  refueling.  On 
the  first  night  of  the  war,  the  Apaches  that  destroyed  the  Iraqi  air  defense  sites  were 
equipped  with  the  ERFS  so  that  the  FARP  could  be  located  as  far  from  the  FLOT  as 
possible. 

As  another  example,  recent  specifications  for  the  US  Army's  next-generation  scout 
reconnaissance  aircraft,  the  RAH-66  Comanche,  imposed  a  stringent  weight  limit  of 
7500  pounds.  This  would  let  the  Comanche  increase  its  self-deployment  range  from 
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800nm,  the  present  maximum  for  scout-attack  aircraft,  to  1260nm.  To  achieve  this  goal, 
however,  many  desirable  information-bearing  avionics  had  to  be  classified  as  "nice  to 
have,"  but  not  as  essential  as  additional  fuel.  Some  years  ago,  such  niceties  included 
HUDs,  ground  mapping  radar,  and  digital  maps.  Now,  they  include  wide  screen  head 
down  displays,  more  capable  mission  computers,  and  integrated  FLIR/ radar  sensors. 
While  such  capabilities  are  routinely  found  on  fixed-wing  tactical  aircraft  (e.g.,  F-15E,  F- 
16C/D),  they  are  often  merely  overly  heavy,  nice-to-have  MEPs  for  helicopters. 

However,  despite  the  chronic  need  to  minimize  rotorcraft  weight,  the  rapid  pace  of 
modularizing  and  miniaturizing  avionics  has  produced  an  explosion  of  information  in 
even  the  most  constrained  cockpits.  Indeed,  such  trends  as  line  replaceable  units 
(LRUs),  line  replaceable  modules  (LRMs),  digital  data  buses,  and  secure  communica¬ 
tions  have  dramatically  increased  information  throughput  to  the  pilot.  This,  in  turn,  has 
placed  more  demands  on  HMD  symbology  design,  to  ensure  that  the  burgeoning 
information  is  presented  and  managed  in  ways  that  do  not  overwhelm  the  pilot's 
limited  perceptual,  cognitive,  and  psychomotor  capacities.  In  particular,  the  avionics- 
induced  shift  toward  decreasing  the  pilot’s  psychomotor  workload  while  simultane¬ 
ously  increasing  the  demands  on  his  cognitive  resources  has  important  implications  for 
one’s  strategy  toward  HMD  symbology  development. 

To  clarify  the  nature  of  the  symbology  issues,  it  is  useful  to  subdivide  the  avionics 
capabilities  into  nine  functional  classes  of  MEPs.  These  are  shown  in  Table  4.1.  This 
classification  is  not  intended  to  represent  the  physical  form  of  the  avionics  in  the 
cockpit,  as  some  MEPs  in  a  given  class  may  reside  in  physically  disparate  locations  (e.g., 
the  different  navigation  subsystems).  Rather,  the  intent  is  to  organize  the  description  of 
MEP  capabilities  along  functional  lines  in  order  to  support  systematic  analyses  of  MEP 
information  requirements. 

It  will  be  important  for  symbology  designers  to  structure  a  general-use  procedure 
for  analyzing  MEP  information  requirements  and  assessing  their  impact  on  HMD 
symbology  design.  Unlike  the  task  analysis  approach,  MEP  analysis  is  not  a  formal 
methodology,  nor  does  it  have  a  rich  empirical  database  on  which  to  draw.  Instead,  the 
design  team  must  go  through  a  series  of  steps — some  analytic,  some  conceptual — to 
determine  the  information  that  is  available  from  the  avionics.  Once  the  available 
information  has  been  identified,  the  team  must  then  use  their  knowledge  of  how  the 
avionics  systems  function  to  determine  which  MEP  component  (e.g.,  navigation  system, 
ASE,  etc.)  will  serve  as  the  source  for  each  required  information  element. 

As  with  other  aspects  of  HMD  symbology  development,  the  key  is  to  characterize 
avionics  functioning  at  the  information  requirement  and  information  element  levels  so  that 
the  optimum  content,  location,  format,  referencing,  and  behaviors  for  HMD  symbols 
may  be  determined.  In  this  analysis,  the  design  team  must  resist  the  temptation  to 
make  engineering-only  assessments  which  stress  electronics  and  "black  boxes"  to  the 
exclusion  of  display  information  that  must  be  perceived,  processed,  and  acted  upon  by 
the  pilot. 

Ideally,  the  MEP  avionics  support  the  pilot's  information  requirements  as  defined 
during  task  analyses.  As  the  MEP  for  a  given  aircraft  becomes  more  defined,  the  design 
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team  can  begin  to  consider  how  well  the  chosen  avionics  satisfy  the  aviator's  informa¬ 
tion  needs.  An  MEP  analysis  should  use  the  results  from  the  information  requirements 
analysis  to  determine  the  avionics  that  can  serve  as  the  source  for  each  information 
element.  The  primary  purpose  is  to  identify  those  required  information  elements  for 
which  the  proposed  MEP  configuration  do  not  provide  a  suitable  avionics  source. 


Table  4.1. 

Classification  of  MEP  Functions  for  US  Army  SCAT  Rotorcraft 


MEP  Functional  Class 

Classification  Rule 

1. 

Flight  Control 

Processing  of  flight  control  data  (e.g.,  altitude,  velocity)  for  other  uses 
(e.g.,  automated  hover  hold) 

2. 

Sensors 

Electronic  and  display  augmentation  of  pilot  vision  (e.g..  Laser  Range 
Finder,  FLIR,  terrain  following  radar) 

3. 

Weapons  Management 

Recording  and  display  of  planning-related  weapons  data  (e.g.,  range, 
number,  trajectory) 

4. 

Target  Acquisition 

Equipment  and  display  aids  to  detect,  acquire,  and  engage  targets 
(e.g.,  laser  spot  tracker,  automatic  target  recognizer) 

5. 

Navigation 

Position-location,  position-updating,  and  route  selection  equipment 
(e.g.,  Doppler  radar,  map,  INS.  GPS) 

6. 

Aircraft  Survivability 
Equipment  (ASE) 

Threat  detectors  (e.g.,  RWR),  threat  identifiers,  missile  jammers  or 
evaders 

7. 

Voice  Communication 

Radio  transmission  and  reception  of  airborne  or  ground-based  voice 
messages  (e.g.,  UHF,  VHF,  HF  radios) 

8. 

Data  Link 

Reception,  storage,  and  transmission  of  airborne  or  ground-based 
digital  data  (e.g.,  ATHS) 

9. 

Battlefield  Awareness 

External  systems  to  provide  battlefield  information  (e.g..  Joint  tactical 
information  distribution  system  [JTIDS],  EPLRS)  on  a  real-  or  near 
real-time  basis 

Analytic  Approach  to  MEP  Evaluation 

The  design  team  can  use  several  different  analytic  methods  to  link  the  avionics 
sources  to  required  information  elements.  One  way  is  to  employ  a  spreadsheet 
approach  like  that  depicted  in  Table  4.2.  For  this  example,  we  have  taken  selected 
required  information  elements  from  the  TAWL  ORDIR  analysis  performed  by  Anacapa 
Sciences  (Hamilton,  Bierbaum,  &  Rogers,  1994).  In  particular,  the  left  column  of  Table 
4.2  displays  seven  of  the  required  information  elements  from  the  Select  Ambush  Posi¬ 
tion  function  of  the  Target  Acquisition  mission  segment.  The  two  right-most  columns 
indicate  the  task  and  information  requirement  associated  with  each  element. 

For  this  analysis.  Columns  2-4  are  of  greatest  concern.  Column  2  lists  the  generic 
source  for  the  information  based  on  the  TAWL  ORDIR  analysis.  As  the  MEP  becomes 
more  defined,  the  design  team  will  replace  the  generic  source  with  a  more  specific 
avionics  source  in  Column  3.  As  can  be  seen,  specific  sources  are  identified  for  the  first 
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four  elements  in  the  table.  Thus,  the  availability  of  an  electronic  aeronautical  chart, 
FLIR,  GPS,  and  die  mission  computer  will  serve  as  sources  for  the  kill  zone,  aircraft 
altitude,  terrain  topology,  and  current  line  of  sight  (LOS)  with  the  target  area. 


Table  4.2. 

Identifying  Specific  MEP  Sources  for  Required  Information  Elements 


Information 

Element 

Generic 

Source 

Specific 

Source 

i 

If  N/A 

Task 

Requirement 

Annotated 

aeronautic 

chart 

Identify 

Kill  Zone 

Where  is  the  kill 
zone  located? 

Altitude  - 
Relative  to 
surrounding 
terrain 

Nav  System 
&  Visual 
Scene  Sensor 

GPS  and 
FLIR 

Determine 
line  of 
sight 

Is  there  inter¬ 
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mizes  cover  for  a  given 
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tion's 
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Map 

N/A 

Aero  chart  lacks  DTED  and 
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determine  pos.  that 
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a  given  A/C  AGL  -  tgt 

MSL  combination 
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cover  and 
concealed 
position 

Which  position 
has  best  cover  & 
concealment? 

On  the  other  hand,  the  bottom  three  information  elements  do  not  have  specific 
avionics  sources  for  optimizing  LOS,  cover,  and  concealment.  In  this  case,  we  have 
assumed  that  while  the  avionics  suite  provides  a  two-dimensional  aeronautical  chart,  it 
does  not  have  a  three-dimensional  digital  map  with  associated  DTED  and  DTED-based 
computations.  For  these  elements,  a  not  available  (N/A)  indication  is  placed  in  Column 
3.  In  Column  4,  we  provide  descriptive  information  that  the  design  team  would  use  to 
summarize  the  reasons  why  information  is  not  available  and  what  would  be  required  to 
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obtain  it.  These  descriptions  can  be  collected  over  the  entire  array  of  information 
elements  and  MEP  possibilities,  and  applied  to  subsequent  steps  of  the  analysis. 

While  the  systematic  application  of  an  analytic  approach  such  as  that  that 
described  here  is  typically  unusual,  the  practice  of  comparing  the  MEP  contributions 
with  the  pilot's  information  requirements  is  crucial  to  both  head-down  and  head-up 
HMD  symbology  definition.  The  number  of  unmet  information  requirements  must  be 
minimized  and  the  impact  of  new  information  elements  and  symbols  imposed  by  more 
powerful  avionics  systems  must  be  identified  as  early  as  possible. 

Conclusions 

For  the  purposes  of  this  project,  we  have  not  attempted  to  identify  and  list  a  series 
of  new  symbols  based  upon  some  predicted  MEP  for  a  future  helicopter.  Instead, 
knowing  that  we  cannot  foresee  all  of  the  important  variables,  we  have  chosen  to 
closely  examine  what  we  believe  to  be  a  collection  of  new  symbols  that  are  broadly 
representative  of  classes  of  new  symbology  types  expected  to  emerge  in  the  near  future. 
Furthermore,  we  have  attempted  to  develop  a  flight  simulator  expressly  designed  for 
the  evaluation  of  anticipated  as  well  as  unforeseen  symbology  types.  The  following 
section  of  the  report  describes  the  development  of  this  simulator. 
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Section  5:  PRISMS  Design  and  Development 


Introduction 

The  Need  for  a  Simulator 

At  the  close  of  the  Phase  I  effort,  the  combined  results  of  the  information 
requirements  analyses,  the  simulator  flights,  and  the  SME  interviews  had  led  us  to 
conclude  that  there  were  a  great  many  ways  that  intelligent  symbology  management 
might  be  incorporated  to  aid  pilots  in  performing  flight  and  mission  tasks.  The 
mission  information  analyses  of  Phase  II  greatly  strengthened  this  conclusion.  Fur¬ 
thermore,  we  recognized  that  the  "innovative  intelligent  symbology  management 
system"  should  be  a  system  that  would  be  applicable  beyond  any  one  specific  aircraft. 
Thus,  although  we  had  chosen  to  focus  on  the  characteristics  of  the  AH-64A  aircraft 
because  of  the  availability  of  SMEs,  the  products  of  the  research  needed  to  be 
extendible  beyond  the  AH-64  and  its  specific  equipment. 

It  became  evident  that  the  extended  applicability  of  our  products  should  take 
two  different  forms:  first,  a  broad  relevance  of  research  results  to  similar  situations 
on  different  aircraft  and  second,  a  continuing  value  of  the  research  methodology 
itself  in  identifying  opportunities  for  intelligent  symbology  management.  We  have 
identified  well  over  one  hundred  information  elements  that  might  be  displayed  as 
new  symbols  on  the  HMD  to  the  benefit  of  the  pilot.  Each  of  these  would  require 
additional  research  in  defining  the  specific  information  content,  format,  behavior, 
and  intelligent  moding.  Only  a  small  fraction  of  the  required  research  could  be 
addressed  within  the  scope  of  this  contract. 

In  accordance  with  these  beliefs,  the  thrust  of  our  Phase  II  project  was  not  only 
aimed  at  providing  some  important  new  HMD  symbols  and  sets  of  rules  for  their 
presentation  or  decluttering,  but  also  at  providing  a  sophisticated  simulator  system 
and  a  reliable  operational  methodology  for  the  continued  use  of  the  rulebase  in 
discovering  and  implementing  intelligent  symbology  management  opportunities. 
This  section  of  the  report  further  describes  the  simulator  design  philosophy  and  the 
way  in  which  it  grew  throughout  the  project. 

STRATA  Simulator  "Flights" 

As  described  in  Section  2,  preparation  for  the  conduct  of  the  SME  interviews 
during  the  Phase  I  effort  included  use  of  the  sophisticated  STRATA  (Simulator 
Training  Research  Advanced  Testbed  for  Aviation)  simulator  operated  by  the  U.S. 
Army  Research  Institute  Aviation  R&D  Activity  (ARIARDA),  at  the  U.S.  Army 
Aviation  Center  at  Fort  Rucker,  Alabama.  Although  it  was  designed  as  a  reconfigu- 
rable  research  testbed,  STRATA  also  provides  all  the  necessary  capabilities  for  an 
outstanding  AH-64  simulator. 
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STRATA  has  a  database  management  workstation,  equipped  with  a  special 
interface  for  experimenters  (or  mission  planners)  to  enter  new  mission  data.  A 
simple  mission  might  involve  navigation  through  a  series  of  waypoints.  More 
complex  scenarios  would  include  wind,  bad  weather,  target  acquisition,  and 
engagement.  Because  our  activities  were  focused  on  the  IHADSS  and  PNVS  sym¬ 
bology  issues,  the  STRATA  capabilities  were  specifically  configured  so  that  the 
project  Principal  Investigator  could  experience  the  IHADSS  display  symbology 
under  a  variety  of  realistic  visibility  conditions,  and  during  a  range  of  flight  and 
mission-oriented  tasks. 

STRATA,  with  an  expert  instructor  in  the  second  seat,  was  remarkably  effective 
in  illustrating  symbol  behavior  in  the  Hover,  Bob-up,  Transition,  and  Cruise  modes. 
A  series  of  coached  flight  maneuvers  and  activities  clarified  critical  HMD  symbology 
issues,  and  demonstrated  the  important  interactions  of  various  symbol  elements. 
Although  the  Principal  Investigator  had  been  very  familiar  with  the  IHADSS 
symbology  from  study  of  the  AH-64  manuals  as  well  as  from  interviews  with  AH-64 
pilots,  the  increased  cognizance  of  symbology  dynamics,  interactions,  features  and 
problems  after  flying  the  STRATA  was  little  less  than  startling. 

As  previously  described,  during  Phase  I  of  the  effort,  we  had  developed  a  laptop 
simulator  designed  to  mimic  the  appearance  and  disappearance  of  Apache  HMD 
symbol  elements  based  upon  rule  "firing"  so  as  to  demonstrate  and  test  the  effects  of 
an  intelligent  symbology  manager  with  Apache  instructor  pilots.  The  system  was 
useful  in  demonstrating  symbol  appearance  and  movement,  but  could  not  accu¬ 
rately  simulate  the  dramatic  effects  of  pilot  head  movement  or  aircraft  attitude  and 
movement  over  the  earth  on  symbol  behavior. 

In  contrast,  the  STRATA  flights  enabled  us  to  observe  symbology  mode 
changes  during  a  variety  of  flight  maneuvers  while  closely  observing  the  behavior 
of  the  HMD  symbology.  We  found  that  it  is  difficult  to  appreciate  the  complex 
dynamics  and  interactions  of  HMD  symbols  without  observing  them  in  action  while 
moving  the  head  to  different  angles  and  attitudes.  The  insights  gained  from  a  few 
hours  use  of  STRATA  dramatically  demonstrated  the  ability  of  a  high-end  simula¬ 
tor  to  clarify  the  symbol  dynamics  and  interactions  and  their  meanings.  Further¬ 
more,  the  results  of  symbology  discussions  with  pilots  were  vastly  more  fruitful 
given  the  opportunity  for  both  of  the  discussants  to  simultaneously  observe  and 
comment  upon  the  symbology  set  in  action. 

From  the  STRATA  experience  it  became  evident  that  the  use  of  a  sophisticated 
simulator  would  greatly  enhance  the  quality  of  the  studies  that  we  could  perform 
and  would  aid  in:  integrating  knowledge  from  the  research,  engineering,  and  pilot 
communities;  demonstrating  established  problems  and  their  candidate  solutions; 
and  applying  the  range  of  research  tools  in  the  most  meaningful  ways. 

The  advantages  of  such  powerful  simulators,  however,  are  accompanied  by 
well-known  disadvantages.  In  general,  they  are  extremely  expensive  to  construct 
and  maintain,  require  a  team  of  specialized  technical  support  personnel,  are  time- 
consuming  to  reprogram  for  new  applications,  and  are  certainly  not  portable  to 


5-2 


other  locations  where  expert  pilots  might  be  permanently  or  temporarily  located. 
These  simulators  are  also  in  great  demand  for  major  research  projects,  such  as  the 
LHX  studies  of  past  years  and  the  current  Air  Warrior  research  efforts.  Despite  the 
surge  of  interest  in  HMD  display  systems,  it  is  extremely  difficult  for  most  research 
groups  to  obtain  access  to  these  costly  and  tightly  scheduled  simulators. 

We  believed  that  the  only  solution  to  these  problems  was  in  the  development 
of  a  sophisticated,  but  relatively  low-cost  HMD  research  simulator,  taking  advantage 
of  the  most  recent  advances  in  technology.  Such  as  system  would  be  so  inexpensive 
as  to  cost  only  1%  of  current  high-end  simulators.  It  would  be  easy  to  reprogram, 
with  a  point-and-click  graphical  user  interface.  It  would  require  no  specialized  skills 
to  design  and  conduct  studies  in  a  timely  manner.  It  would  be  easily  transportable 
for  use  at  field  sites.  Finally,  because  of  its  revolutionary  cost-performance  ratio,  it 
would  provide  a  much  broader  segment  of  the  research  community  with  the  simu¬ 
lation  capabilities  necessary  for  the  experimentation  so  urgently  needed  for  the 
development  of  new  flight  systems. 

The  following  pages  provide  an  overview  of  the  PRISMS  design  and  develop¬ 
ment  process.  The  specific  descriptions  of  the  PRISMS  features  and  interface  are 
provided  in  the  system  User's  Manual. 

PRISMS  Design  Analyses 

Since  the  beginning  of  the  project,  and  continuing  throughout  its  course,  the 
Principal  Investigator  at  Anacapa  Sciences  and  the  System  Developer  at  Thought- 
Wave  conferred  almost  daily  to  plan,  define,  expand,  and  refine  the  PRISMS  capa¬ 
bilities.  Early  on,  a  preliminary  set  of  PRISMS  II  design  requirements  was  furnished 
to  ThoughtWave  to  clearly  identify  the  system  goals  from  the  user's  perspective. 
These  requirements  included  the  specific  required  capabilities  of  PRISMS,  proce¬ 
dures  and  capabilities  needed  for  demonstrations,  system  preparation  requirements 
for  experimental  research,  means  for  definition  of  individual  symbols  and  symbol 
sets,  selection  of  terrain  characteristics,  and  requirements  for  point,  linear,  and  area 
annotations  of  the  digitized  terrain  elevation  data. 

ThoughtWave  responded  with  an  initial  effort  to  identify  the  hardware  and 
software  components  that  would  deliver  a  PRISMS  of  the  highest  possible  quality 
within  time  and  budget  constraints.  This  objective  required  iterative  trade-off 
analyses  of  functional  requirements;  vendor  system  capabilities,  costs,  and  potential 
delivery  dates;  and  all  of  the  issues  of  component  compatibility.  In  fact,  the  review 
and  analysis  was  to  continue  for  months  into  the  project  as  new  products  became 
available  and  prices  for  others  tumbled. 

Over  time,  and  after  hours  of  telephonic  and  face-to-face  meetings,  the  Princi¬ 
pal  Investigator  learned  a  great  deal  about  the  difficulties  of  system  integration  and 
the  unsuspected  shortcomings  of  available  hardware  and  software  products.  In  like 
manner,  the  System  Developer  began  to  appreciate  the  unusual  design  require¬ 
ments  of  an  experimental  psychologist  attempting  to  prepare  for  a  potentially 
enormous  realm  of  behavioral  and  human  engineering  research  projects. 
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Although  compromises  were  required,  the  goal  of  an  extremely  flexible,  yet  low  cost 
simulator  system  guided  the  team  throughout  the  effort. 

One  of  the  goals  of  the  development  was  to  make  the  simulation  as  realistic  as 
possible,  incorporating  "virtual  reality"  techniques  such  that  the  experience  would 
include  an  "inside-out"  perspective  of  self-directed  flight  through  terrain  landforms 
and  full  head  movement  for  exploration  and  examination  of  surrounding  features. 

Different  techniques  for  inducing  virtual  reality  may  be  judged  on  the  degree  of 
"immersion"  or  sense  of  "presence"  each  provides.  Although  these  are  subjective 
aspects  of  the  experience  and  there  is  no  single  metric  of  "immersion"  many  of  the 
contributing  factors  have  been  identified,  as  shown  in  Figure  5-1.  As  shown  by  the 
checked  items  at  the  right  of  the  figure,  we  determined  that  PRISMS  could  and 
should  incorporate  nearly  all  of  the  key  features  necessary  for  providing  a  fully 
immersive  experience. 


Immersion 

Threshold 

Non-interactive 
Slow  update  rate 
Low  image  complexity 
Single-source  sound 
Screen  display 
Low  resolution 
Monoscopic 
Small  field  of  view 
No  head  tracking 


PRISMS 

Interactivity  V 

Rapid  update  rate  V 

High  image  complexity  V 

3D  sound  system  V 

Head-mounted  display  V 

High  resolution  V 

Stereoscopic 

Large  field  of  view  V 

Head  tracking  V 


Figure  5-1.  Factors  influencing  the  degree  of  virtual  reality  immersion  with  check¬ 
marks  indicating  PRISMS  capabilities  (adapted  from  Pimentel  & 
Teixeira,  1993). 

Desired  Capabilities 

Over  the  course  of  the  project,  we  continued  to  expand  and  flesh  out  sets  of 
sample  experimental  scenarios  in  order  to  clarify  the  PRISMS  design  requirements. 
The  primary  requirements  categories  included  (1)  a  set  of  selectable  simulated  flight 
capabilities  ranging  from  fully  automatic  to  partially  constrained,  to  fully  pilot 
controlled,  (2)  selectable  HMD  symbology  elements  of  a  broad  number  of  types, 
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behaviors,  and  moding  structures,  grouped  and  arranged  as  desired,  (3)  selectable 
capabilities  for  conducting  experiments  and  demonstrations  such  as  "head-down" 
views  of  horizontal  situation  displays  or  other  data,  (4)  environmental  controls 
(sun,  moon,  wind,  haze,  etc.),  and  (5)  target  appearance  and  weaponry  applications 
for  use  in  evaluating  situation  awareness. 

Despite  its  low  cost,  we  also  determined  that  PRISMS  had  to  provide  the  capa¬ 
bilities  necessary  to  show  HMD  symbology  in  screen-fixed,  aircraft-fixed,  and  earth- 
fixed  frames  of  reference  (described  below),  along  with  facilities  for  symbology 
demonstration,  knowledge  acquisition,  experimental  control,  and  extensive  data 
recording.  In  addition,  and  within  cost  constraints,  many  high-end  features  were  to 
be  included,  such  as  a  gaming  area  of  realistic  terrain,  multiple  moving  targets, 
photo-textured  objects,  3D  sound,  and  voice  synthesis  and  recognition. 

In  addition  to  the  helicopter  cyclic,  collective,  and  pedal  controls  for  full  flight 
control  by  experienced  pilots,  it  was  deemed  valuable  to  permit  flight  by  non-pilots, 
such  as  offering  flight  along  a  pre-determined  path,  as  if  on  an  "invisible  rail,"  for 
demonstrations  or  experiments  not  requiring  aircraft  control  inputs.  Between  these 
two  extremes,  any  level  of  error  constraint  could  be  invokable,  such  as  flight  within 
an  "invisible  tube"  of  experimenter-controlled  diameter  for  use  in  studies 
employing  beginning  or  intermediate-level  pilots. 

Physical  Configuration 

Major  Components 

The  two  PRISMS  shipping  cases  contain  a  cockpit  station,  an  experimenter's 
station,  and  an  electronics  station.  The  cockpit  station  includes  the  pilot's  seat,  full 
flight  controls,  HMD  with  headphone  and  microphone,  and  head-tracking  system. 
The  experimenter's  station  includes  a  large-screen  monitor  (with  a  second  monitor 
optional),  keyboard  and  mouse,  four-axis  flight  control,  headphone  and  micro¬ 
phone,  and  the  symbology  management  system.  The  electronics  station  includes 
two  dual-CPU  NT  workstations,  three  OpenGL  graphics  accelerators,  a  six-channel 
audio  mixer,  a  four  channel  audio  amplifier,  a  full  matrix  video  switching  system, 
and  a  VGA  to  NTSC/PAL  converter  for  video  recording  of  PRISMS  sessions. 

Anthropometric  Study 

In  planning  the  simulator  physical  configuration,  an  anthropometric  study  was 
completed  to  identify  the  range  of  dimensions  necessary  to  accommodate  test  subject 
pilots  and  to  meet  typical  control  locations  relative  to  human  body  dimensions. 

The  control  positions  were  adapted  from  MIL-STD-1333A,  Aircrew  Station 
Geometry  for  Military  Aircraft.  Although  this  document  assumes  seat  adjustability, 
PRISMS  was  designed  to  use  pedal  position  adjustability  to  provide  the  same 
anthropometric  dimensional  range  at  much  lower  cost  and  weight.  The  reach  enve¬ 
lopes  for  all  controls  are  within  the  comfortable  zones  for  5%  through  95%  male 
operators. 
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Figure  5-2.  The  PRISMS  system,  nearing  completion. 
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Polhemus 
10"  range 


Collective  and  rudder  control  positioning 

(adapted  from  MIL -STD-1 33  3 A,  which  assumes  seat  adjustability) 

#  Seat  back  width  is  about  18"  and  shall  not  impede  elbow  movements  to  rear  by  either  arm 
0  A  single  heel  point  height  (e.g.,  9"  below  SFP)  is  acceptable  although  5"  range  preferable 
f  Heel  point  adjust  ability  range  30  to  40"  from  SRP  is  required  for  5%  to  95%  pilot  dimensions 
£  FLidder  pedal  fore-aft  range  of  movement  is  6"  (Thrustmaster  OK) 


0  Collective  full-up  grip  location  a  maximum  of  8"  above  SRP 

£  Collective  full-down  location  is  firmly  set  at  SRP  level 

^  Seat  pan  angle  (thigh  tangent  angle)  1 0-20°  acceptable 
(If  1  3°,  it  will  be  at  90°  to  back  for  ease  of  packing) 

£  Cyclic  grip  reference  point  is  1  0-11  inches  above  SRP 

9  Seat  back  angle  of  13°  is  preferred 


11" 


Seat  pan 
Collective  approx 
grip  full  up  16"  long, 

\18"  wide, 
10°  angle 

no-11"  YV  7.5" 


13° 


5.5" 


rSRP 


k  i 

i  Collective 

6"  grip  full  down 

L _ 

M  13"  * 

‘//  ******  *  *  ///j 

:;;Heel  Point  '/A 
v /accept  able/// 
\  Y/s  range  '/A 

30"  "" 

_ w. 

40" 

26" 


5" 


t  t 


Collect  ive 
‘pivot  point 


The  Polhemus  device  would  be  mounted  on  the  box  behind  t  he  seat  back. 

To  accomodate  1  -99%  male  sitt  ing  eye  height  (28"  to  34.25")  plus  some  elevation 
for  head  tilting,  minus  some  elevation  for  the  "slouch"  factor,  a  range  of  26"  to 
36"  abovethe  compressed  seat  cushion  top  isrecommended. 


Seat  back 
approx 
16"  tall,  18" 
wide  with 
13°  angle 


Figure  5-3.  Example  of  anthropometric  criteria  and  dimensions  for  PRISMS. 

Packaging  and  Transportability 

Cockpit  design  efforts  focused  on  providing  a  realistic  and  effective  control 
location  geometry  for  the  cyclic,  collective,  and  pedals,  while  permitting  the  system 
to  be  rapidly  packed  or  unpacked  and  set  up.  The  design  objectives  included  the  use 
of  shipping  cases  that  permit  quick  access  to  seat  and  control  mechanisms  and 
storage  for  the  HMD,  CPUs,  hard  disks,  and  the  associated  computer  components. 
The  test  subject's  seat  itself  serves  as  a  storage  area  in  order  to  save  weight  and  space. 
The  total  weight  of  the  two  packed  cases  is  now  several  hundred  pounds.  This  is 
considerably  more  than  was  anticipated  at  the  beginning  of  the  project,  but  still 
allows  for  reasonably  convenient  transporting  to  conferences,  military  facilities,  and 
other  meeting  sites  for  clearly  communicating  symbology  behavior  and  conducting 
knowledge  acquisition  sessions  with  subject-matter  experts  (SMEs). 

Flight  Controls 

Because  the  high-end  military  flight  simulation  controls  were  known  to  cost 
thousands  of  dollars  each,  we  selected  the  best  available  gaming  system  hardware. 
We  purchased  a  set  of  ThrustMaster  controls  including  the  F-16  FLCS  Limited 
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Edition  control  stick.  Weapons  Control  System  (WCS),  and  the  Rudder  Control 
System  (RCS).  These  served  respectively  as  the  cyclic,  collective,  and  pedals  of  the 
PRISMS  simulator. 


Figure  5-4.  The  ThrustMaster  WCS,  RCS,  and  FLCS. 

The  six-inch  control  arm  of  the  WCS  was  found  to  be  too  short  for  the  sensitive 
collective  inputs  desired  by  our  pilots,  and  was  modified  to  approximately  15  inches. 
Although  the  modification  worked  well,  the  extended  lever  arm  resulted  in  exces¬ 
sive  wear  and  tear  on  the  lightweight  components,  so  the  WCS  was  replaced  by 
another,  very  sturdy  collective  with  a  stainless  steel  arm  built  by  Flight  Link.  This 
control  offers  a  very  smooth  twist-grip  handle  for  rpm  control,  should  this  be  neces¬ 
sary  in  a  simulated  aircraft. 


Figure  5-5.  The  Flight  Link  helicopter  collective  control. 

The  FLCS  was  found  to  be  quite  satisfactory  except  for  the  high  displacement 
force  requirements.  The  FLCS  was  modified  to  reduce  the  forces  to  levels  more 
representative  of  helicopter  controls.  New  torsion  springs  were  installed,  reducing 
the  maximum  required  forces  from  nearly  50  pounds  to  a  level  just  sufficient  to 
return  the  stick  to  center  position. 
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Figure  5-6.  A  close-up  showing  the  FLCS  in  use  as  the  PRISMS  cyclic. 

Helmet-Mounted  Display  Selection 

The  selection  of  an  HMD  for  PRISMS  use  was  one  of  the  more  difficult  issues. 
First  of  all,  helmets  vary  across  a  broad  range  of  parameters  including  horizontal 
and  vertical  field  of  view,  brightness,  color  capabilities,  video  formats,  pixel  size  and 
number,  weight,  and  many  other  factors.  Secondly,  the  range  of  costs  is  quite  broad; 
roughly  $1,000  to  $120,000.  After  extensive  deliberation,  and  a  visit  to  Kaiser  Electro- 
Optics  in  Carlsbad,  CA  for  demonstrations,  we  arranged  to  purchase  the  Kaiser 
lOOOHRpv  VIM  (Visual  Immersion  Module)  Personal  Viewer  for  use  with  PRISMS. 
The  VIM  provided  a  30°  vertical  by  100°  horizontal  field  of  view  with  four  full-color 
0.7  in.  active  matrix  LCDs  (180,000  pixels  per  LCD).  The  VIM  was  lightweight  (26  oz), 
was  eyeglasses  compatible,  and  supported  high  fidelity  stereo  sound.  The  head¬ 
tracking  was  provided  by  a  built-in  Polhemus  INSIDETRAK  system  with  resolution 
of  0.03°  in  orientation. 


Figure  5-7.  The  Kaiser  VIM  Personal  Viewer  helmet. 

However,  some  months  after  placing  our  order,  we  were  informed  by  Kaiser 
Electro-Optics  that  the  VIM  helmet  was  no  longer  being  made.  Instead,  we  were 
given  an  opportunity  to  review  the  characteristics  and  costs  of  the  new  Pro  HMD 
from  Kaiser.  Unfortunately,  the  lowest-priced  model  offered  a  field  of  view  of  only 
18°  by  24°  —  unsuitable  for  our  purposes.  The  cost  of  a  Kaiser  helmet  with  a  wide 
field  of  view  similar  to  the  VIM  would  have  exceeded  the  PRISMS  budget. 

After  an  extensive  search,  we  were  able  to  identify  an  alternative  LCD  device, 
the  Visette  Pro,  manufactured  by  Virtuality  and  marketed  in  the  USA  by  nVision. 
The  Visette  Pro  is  very  sturdy  and  reliable,  incorporates  its  own  Polhemus  Inside- 
Trak  system  and  provides  a  60°  by  47°  field  of  view. 


Fig  5-8.  The  Virtuality  Visette  Pro  helmet. 

We  arranged  for  a  demonstration  of  the  device  and  were  favorably  impressed 
by  the  quality  of  the  imagery,  especially  considering  the  relatively  low  price,  and 
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ordered  a  system.  Since  that  time  we  have  also  used  three  of  the  nVision  HMD 
systems  with  good  effect,  including  the  Datavisor  VGA,  the  Datavisor  VGA/HiRes 
(up  to  a  78°  field  of  view),  and  the  DatavisorLCD  (with  a  60°  field  of  view).  As  we 
observed  the  rapid  technological  and  cost  changes  occurring  in  the  HMD  field,  we 
were  careful  to  design  PRISMS  to  permit  the  use  of  any  HMD. 

Head-Tracking 

One  of  the  PRISMS  goals  was  to  offer  an  "immersive"  approach,  providing  an 
effective  virtual  reality  experience  for  more  realistic  representation  of  military 
rotorcraft  tasks.  In  response  to  the  pilot's  control  inputs,  the  PRISMS  aircraft  alti¬ 
tude,  attitude,  heading,  and  speed  change  as  they  would  in  a  real  helicopter.  But  the 
addition  of  an  opaque  visor  on  the  HMD  and  accurate  head-tracking  fully  involves 
the  pilot  in  the  simulation.  As  the  pilot  turns  his  head  and  shoulders,  the  field  of 
view  moves  through  the  full-surround  field  of  regard  and  additional  portions  of  the 
aircraft  or  the  terrain  come  into  view.  The  effect  of  being  fully  surrounded  by 
simulated  terrain  is  enormously  more  realistic  and  immersive  than  peering  at  a 
monitor  on  a  desktop.  Thus,  the  goal  of  immersiveness  required  an  accurate  head 
tracker  so  that  symbology  positioning  and  behavior  is  appropriately  slaved  to  the 
user's  head  movements. 

An  additional  important  use  of  the  head-tracker  is  that  of  a  look-down  capabil¬ 
ity  for  the  HMD  imagery  that  presents  images  in  the  cockpit  area  when  the  pilot 
rotates  his  head  downward,  as  if  to  look  at  the  console  or  his  lap.  Maps,  HSDs,  other 
avionics  displays,  or  kneeboard  data  may  be  presented  to  the  pilot  in  this  manner. 
When  the  experimenter  or  trainer  is  developing  a  new  session,  various  set-up  tools 
may  be  presented  to  aid  in  selection  of  test  parameters.  The  head-down  capability  is 
particularly  useful  in  permitting  presentation  of  any  scanned  topographic  map  as  a 
head-down  display  in  the  PRISMS  cockpit.  This  addition  adds  realism  to  the 
situation  awareness,  navigation,  or  other  experimental  tasks  performable  in  the 
simulator.  If  desired,  and  appropriate  scaling  and  rectification  values  are  entered,  an 
aircraft  present  position  "bug"  could  be  also  be  shown  on  the  pilot's  map,  simulating 
a  digital  map  system. 

Prior  to  the  purchase  of  the  HMD,  we  used  a  Polhemus  InsideTrak  to  evaluate 
it's  characteristics  and  found  them  quite  suitable  for  PRISMS.  When  the  Visette  Pro 
helmet  was  purchased,  it  was  furnished  complete  with  an  imbedded  InsideTrak 
sensor.  The  Visette  Pro  head  tracking  sensor  was  calibrated  using  the  latest  Inside¬ 
Trak  driver  for  WorldToolKit  under  NT  4.0.  The  unexpected  difficulty  of  the 
calibration  was.  a  frustrating  impediment  to  the  accurate  integration  of  the  flight 
model  with  the  HMD  field  of  regard.  A  number  of  factors,  including  the  unusual 
mounting  position  and  angle  of  the  sensor  led  to  the  difficulties.  Nevertheless, 
iterative  refinements  reduced  the  calibration  error  to  an  undetectable  level. 

PRISMS  is  also  designed  to  use  other  head-tracking  systems  such  as  the  Ascension 
Flock  of  Birds. 
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Figure  5-9.  The  PRISMS  HMD  view  of  a  map  on  the  aircraft  console. 


Experimenter  Station 

As  the  pilot  flies,  the  experimenter  or  trainer  is  able  to  view  several  different 
movable  and  resizable  windows  on  his  monitor.  As  shown  in  Figure  5-10,  the 
experimenter  can  simultaneously  observe  the  pilot’s  HMD  view,  maps  in  various 
scales  with  present  position  indicators,  and  dialog  windows  designed  for  controlling 
session  settings,  events,  or  recordings.  This  context  is  similar  to  flying  with  the  pilot 
in  a  test  aircraft  without  the  cost  or  safety  concerns  usually  associated  with  test 
flights,  and  permits  discussions  of  flight  symbology  during  realistic,  representative 
flight  operations. 

Furthermore,  the  experimenter  or  trainer  can  stop  the  action,  query  the  pilot 
on  his  actions  or  impressions,  and  replay  a  flight  sequence  after  introducing  a 
change  in  symbology.  A  series  of  scenarios  can  be  predefined  and  linked  together  to 
permit  a  broad  variety  of  tactical  situations,  aircraft  maneuver  requirements,  and 
pilot  activities  in  a  brief  time  period.  PRISMS  is  designed  for  flexibility  of  applica¬ 
tion  so  that  individual  researchers  will  find  it  easy  to  incorporate  and  demonstrate  a 
wide  variety  of  symbology  ideas  and  easy  to  prepare  for  more  formally  designed 
experiments. 
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Session  Control  Edit  Mules  Categories  About  Prisms 


Figure  5-10.  An  example  of  windows  available  on  the  experimenter’s  monitor. 


A  present  position  “bug”  can  be  displayed  on  the  experimenter’s  station  map, 
an  invaluable  aid  for  monitoring  the  pilot’s  progress  through  the  terrain  and  his 
current  heading  and  position. 

The  provision  of  a  second  aircraft,  controlled  with  a  joystick  from  the  experi¬ 
menter’s  station,  was  undertaken  to  lend  additional  realism  to  an  already  immer¬ 
sive  environment.  The  second  aircraft  can  be  used  as  a  wingman  or  an  enemy 
aircraft,  or  simply  as  another  “eye  position”  to  view  the  exterior  of  the  pilot’s 
aircraft.  Object  and  attribute  passing  using  inter-process  communication  was 
implemented  to  allow  each  of  the  two  helicopters  to  update  the  other’s  position. 

Workstation  Processors 

PRISMS  incorporates  two  dual  CPU  NT  workstations.  The  Intergraph  TDZ 
2000  was  tested  at  its  introduction  at  COMDEX  in  1998.  The  tests  showed  that  it 
would  approximately  triple  the  simulation  frame  rates  we  had  previously  achieved 
in  the  pilot’s  station.  Depending  upon  the  terrain  models  visible  and  the  density  of 
the  vegetation  texture,  the  frame  rate  ranges  as  high  as  70  Hz.  Frames  rates  are 
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dependent  upon  a  number  of  factors  such  as  the  extent  of  terrain  visible  (number  of 
model  areas  depicted),  the  types  of  vegetation  codes  presented,  and  the  number  of 
symbol  elements  shown. 


Figure  5-11.  A  view  of  the  PRISMS  experimenter’s  station. 

Any  system  can  be  “brought  to  its  knees”  by  increasing  the  density  of  the 
presentation  until  frame  rates  fall  below  acceptable  levels.  For  this  reason,  the 
PRISMS  interface  permits  the  experimenter  to  make  the  appropriate  trade-offs.  For 
example,  a  linear  symbol  “draping”  algorithm  was  devised  giving  the  experimenter 
indirect  control  over  the  number  of  points  used  to  describe  lines  drawn  on  the 
terrain,  trading  off  line  precision  against  acceptable  frame  rate. 

Software 

Flight  Model 

Although  various  flight  models  are  commercially  available,  we  elected  to 
develop  our  own.  Although  a  difficult  decision,  it  permitted  us  to  maintain  control 
over  the  model’s  complexity  and  the  resulting  impact  on  video  frame  rate. 
Furthermore,  it  assured  that  we  would  have  direct  control  over  variables  such  as 
thrust,  power,  mass,  velocity,  drag,  torque,  angular  velocities  and  moments  of 
inertia,  relative  wind  moments,  and  trim  adjustments,.  Iterative  qualitative 
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evaluations  of  aircraft  handling  during  flight  through  the  New  Mexico  database 
were  conducted  by  the  authors,  as  well  as  Army  helicopter  pilots  with  very  satisfying 
results  and  continual  improvements  to  the  fidelity  of  the  aerodynamic  simulation. 

During  the  summer  of  1997,  Loran  Haworth,  the  Project  COTR  and  Gene 
Kasper,  the  Alternate  Project  COTR  visited  ThoughtWave  headquarters  in  Torrance 
California.  A  briefing  was  provided  on  the  planned  capabilities  of  the  PRISMS 
simulator,  and  demonstrations  of  the  system  symbology,  terrain  presentation  tech¬ 
niques,  and  helicopter  flight  control  model.  Both  the  COTR  and  ACOTR  were  given 
opportunities  to  "fly"  through  simulated  mountainous  terrain  in  New  Mexico, 
using  the  flight  controls  and  the  HMD  with  Apache  symbology.  They  provided  us 
with  a  number  of  useful  recommendations  such  as  reducing  the  cyclic  forces  and 
introducing  some  stability  augmentation  or  autopilot  functions  to  reduce  pilot 
workload. 

Our  Apache  SME,  Mike  Couch,  provided  an  interim  evaluation  of  the  PRISMS 
simulator  handling  over  the  course  of  three  days  at  the  ThoughtWave  offices.  A 
primary  objective  of  these  trials  was  to  maximize  the  similarity  of  the  PRISMS 
handling  to  that  of  the  AH-64.  In  enhancing  the  PRISMS  flight  model  based  on  the 
SME's  suggestions,  improvements  to  the  handling  qualities  were  made  as  quickly  as 
possible,  so  that  several  iterations  could  be  completed  in  the  time  available.  In 
conducting  these  activities,  the  payoff  of  having  of  developed  our  own  flight  model 
was  most  evident.  SME  comments  about  handling  qualities  could  usually  be 
directly  related  to  factors  such  as  mass  or  drag  and  corrections  could  be  made  accord¬ 
ingly. 

A  view  of  the  SME  in  the  PRISMS  cockpit  station  is  shown  in  Figure  5-12.  The 
cyclic  and  pedal  controls  are  visible,  as  is  the  HMD  and  head  tracker.  Components  of 
the  experimenter's  station  are  shown  in  the  background. 

Among  the  adjustments  to  the  flight  model  were  increasing  the  tendency  of 
the  helicopter  nose  to  remain  in  the  direction  of  flight  and  making  the  overall 
handling  qualities  more  forgiving  and  smooth.  Loran  Haworth  again  visited  the 
ThoughtWave  facility  on  the  spring  of  1998  in  order  to  review  the  progress  of  the 
project,  test  the  new  flight  controls  and  handling  qualities  of  PRISMS,  and  observe 
the  recently  introduced  symbols  and  second  helicopter  capability. 

Although  PRISMS  currently  is  configured  to  have  flight  handling  characteris¬ 
tics  similar  to  the  AH-64  Apache,  it  is  important  to  recognize  that  the  flexible  flight 
model  easily  permits  simulation  of  other  helicopters.  Fixed-wing  simulation  or 
ground  vehicle  simulation  are  also  possible  with  relatively  few  changes. 

Symbology  and  Frames  of  Reference 

Because  Apache  pilots  form  the  available  pool  of  attack  helicopter  SMEs,  we 
have  constructed  a  complete  set  of  symbols  from  the  AH-64  Integrated  Helmet  and 
Display  Sighting  System  (IHADSS)  as  a  baseline  to  which  other  symbols  may  be 
added.  Complete  descriptions  of  these  symbols,  their  purposes,  and  their  behaviors 
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are  provided  in  our  Phase  I  final  report  (Rogers,  Spiker,  &  Asbury,  1996).  In  addi¬ 
tion,  PRISMS  provides  a  set  of  tools  for  easily  defining  new  symbols  and  their 
appearances,  behaviors,  and  the  rules  for  their  presentation  or  removal  from  the 
current  symbol  set.  PRISMS  is  also  designed  to  permit  the  use  of  any  of  three  differ¬ 
ent  “frames  of  reference,”  as  described  below. 


Figure  5-12.  Apache  pilot  testing  the  PRISMS  flight  model. 

In  order  to  effectively  describe  and  control  the  behavior  of  HMD  symbols,  three 
different  frames  of  reference  must  be  considered.  First,  symbols  may  be  “screen- 
fixed,”  as  if  painted  on  the  helmet  visor,  so  that  they  are  always  in  the  same  position 
relative  to  the  pilot’s  field  of  view.  Nearly  all  of  the  Apache  symbols  are  of  this  sort. 
Second,  other  symbols  may  be  “aircraft-fixed,”  in  that  the  symbol  is  moved  or 
rotated  to  compensate  for  head  movements  and  stay  in  the  same  position  relative  to 
the  aircraft  itself.  The  Apache  has  one  (diamond-shaped)  symbol  that  behaves  in 
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this  manner.  It  is  called  the  "head-tracker"  and  is  used  to  show  the  relative  position 
of  the  aircraft  nose. 

Third,  symbols  may  be  "earth-fixed,"  in  that  the  symbol  is  moved  or  rotated  to 
compensate  for  both  pilot  head  motion  and  aircraft  motion,  so  that  the  symbols 
appear  to  be  located  on  or  above  positions  in  the  real-world  terrain.  No  earth-fixed 
symbols  are  currently  included  in  the  Apache.  Flight  simulator  studies  of  these 
earth-fixed,  or  "conformal"  HMD  symbols  have  rarely  been  published  in  the  open 
literature.  One  such  study  is  that  of  Haworth  and  Seery  (1992).  Because  the  earth- 
fixed  symbols  are  likely  to  have  a  major  role  in  the  next  generation  of  HMD 
symbology,  PRISMS  was  specifically  designed  to  support  their  simulation. 

Digital  Terrain  Data  Availability 

Although  we  had  initially  anticipated  developing  our  own  terrain  database 
using  USGS  or  other  digital  terrain  elevation  data  (DTED)  and  digital  feature  analy¬ 
sis  data  (DFAD),  we  learned  of  the  progress  of  the  South  West  USA  (SWUSA)  data¬ 
base  from  work  on  another  project.  Discussions  were  then  held  with  the  Data  Base 
Generation  System  (DBGS)  group  of  Lockheed-Martin  at  the  58th  Training  Support 
Squadron  at  Kirtland  AFB  in  Albuquerque  (supporting  mission  preview  simula¬ 
tion).  The  discussions  focused  on  the  possibility  of  acquiring  this  highly  enhanced 
DMA  data  for  a  large  area  in  New  Mexico.  Although  it  was  determined  that  acquisi¬ 
tion  of  the  data  would  be  possible  for  this  project,  it  seemed  unlikely  that  the 
PRISMS  processing  power  would  permit  effective  use  of  the  data  in  a  moving  scene. 
Thus,  we  continued  to  plan  to  use  USGS  data,  successively  adding  gradual  textural 
enhancements  to  minimize  performance  degradation. 

Some  months  later,  we  obtained  a  Multigen  OpenFlight  terrain  database  depict¬ 
ing  the  Hunter-Liggett,  California  area  with  full  texturing  and  high  fidelity  cultural 
feature  models  representing  the  airfield  area.  Because  the  Hunter-Liggett  data  was 
believed  to  be  similar  to  the  Lockheed-Martin  SWUSA  data  we  were  surprised  to 
find  that  we  were  able  to  "fly"  though  the  area  using  head  tracked  control,  while 
viewing  the  terrain  on  a  monitor.  Although  our  original  estimates  suggested  that 
the  processing  power  requirements  for  use  of  the  SWUSA  data  were  too  great  for 
PRISMS,  we  subsequently  determined  that  this  enormous  database  (240,000  square 
miles  of  New  Mexico  terrain)  could  indeed  be  employed. 

The  PRISMS  geographic  location  mechanisms  employed  with  the  SWUSA  data 
base  were  designed  to  include  both  latitude-longitude  and  UTM  coordinate  systems. 
These  mechanisms  permit  the  selection  of  flight  areas  during  an  experimental 
session,  the  definition  of  specific  positions  such  as  waypoints,  and  the  scoring  of 
flight  performance  precision.  The  accuracies  of  the  mechanisms  were  repeatedly 
evaluated  through  analytical  comparisons  with  USGS  paper  map  coordinate  and 
elevation  data.  The  coordinates  of  prominent  terrain  features  in  mountainous 
portions  of  the  New  Mexico  terrain  were  identified  from  USGS  1:24 ,000-scale  paper 
maps.  The  coordinates  were  input  to  PRISMS  and  the  accuracy  of  the  generation  of 
resulting  views  were  evaluated  through  map-terrain  analysis. 
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Metrics 

As  we  began  to  recognize  the  broad  range  of  potential  applications  of  the 
PRISMS  simulator,  we  made  a  concerted  effort  to  include  all  of  the  tools  the 
research  community  might  require.  A  literature  review  was  undertaken  to  identify 
the  most  important  experimental  methods  and  metrics,  so  that  the  necessary  tools 
for  each  type  of  research  would  be  made  available.  A  primary  objective  of  the  inter¬ 
face  design  effort  has  been  to  permit  the  experimenter  to  choose  and  implement 
sophisticated  metrics  in  the  dialog  without  any  requirement  for  additional  software 
programming. 

PRISMS  was  designed  to  incorporate  a  range  of  selectable  performance 
measurements  including  (1)  root  mean  square  error  (RMSE)  from  a  designated 
flight  path,  altitude,  or  airspeed,  (2)  accuracy  of  HMD  reticle  placement  on  a  specified 
feature  in  the  terrain,  (3)  response  time  for  target  detection  and  accuracy  of  target 
engagement,  (4)  total  time  of  intervisibility  with  a  known  enemy  position,  (5) 
waypoint  crossing  accuracy,  (6)  precise  position  landing  accuracy,  and  (7)  evaluating 
pilot  skills  with  aircraft  maneuvers  such  as  the  Aeronautical  Design  Standard  (ADS- 
33)  "pirouette"  with  measures  of  altitude,  heading,  and  distance  errors. 

In  addition  to  these  built-in  measurement  options,  provisions  have  been  made 
to  ensure  the  ease  of  creating  new  metrics  as  needed  by  the  experimenter  or  trainer. 
For  example,  some  new  PRISMS  metrics  were  needed  to  support  the  experiments 
conducted  with  the  Apache  pilots.  The  dialog  structure,  with  its  provisions  for 
creating  rules,  expressions,  and  operations,  was  used  to  construct  rules  that  would 
start  and  stop  timers  for  measuring  head  azimuth  dwell  times  in  nine  segments  of 
the  forward  180°. 

User  Interface 

A  large  portion  of  the  system  design  activities  was  directed  at  user  interface 
development.  One  of  the  primary  objectives  for  PRISMS  was  that  this  powerful 
system  be  easily  operable  without  need  for  sophisticated  programming  skills. 
Because  the  system  is  very  versatile  and  the  potential  users  may  have  countless 
ideas  for  demonstrations  and  experiments,  great  flexibility  was  required  of  the 
dialog.  The  challenge  was  in  devising  a  dialog  that  could  provide  this  flexibility 
without  adversely  impacting  its  ease  of  use. 

In  addition  to  frequent  telephone  discussions,  many  extended  meetings  were 
held  at  ThoughtWave  and  Anacapa  facilities  for  firming  up  the  philosophy  for  the 
experimenter-computer  dialog  structures.  The  design  philosophy  has  attempted  to 
balance  an  almost  open-ended  range  of  PRISMS  capabilities  with  easy-to-use 
features  for  constructing  and  conducting  experiments  and  demonstrations. 

The  initial  intent  in  the  development  of  the  PRISMS  dialog  was  to  provide  a 
broad  range  of  powerful  simulation  and  experimentation  capabilities  yet  make  the 
interface  so  intuitive  and  forgiving  that  it  would  require  only  pointing  and  clicking 
on  button  selectors,  check-boxes,  and  entering  a  few  text  and  numeric  items. 
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However,  as  the  PRISMS  system  capabilities  grew,  the  number  of  possible  selections 
made  such  an  interface  unwieldy.  Finally,  with  the  decision  to  allow  the  PRISMS 
user  to  construct  essentially  any  type  of  tasks,  conditions,  and  metrics,  the  notion  of 
a  simple,  unchanging  graphic  interface  became  wholly  inapplicable.  Instead,  what 
was  required  was  a  more  powerful,  expandable  dialog  approach  that  could 
nevertheless  be  quickly  mastered  by  a  researcher  without  formal  training  in 
programming. 

It  also  became  apparent  to  us  that  the  interface  should  include  multiple  levels 
of  user  interaction.  At  the  highest  level,  a  user-friendly  interface  should  permit  the 
selection  of  parameters  for  most  experimental  requirements,  such  as  choosing  the 
HMD  symbols  to  be  included  and  basic  metrics  to  be  employed.  A  series  of  pop-up 
and  pull-down  menus  and  screens  has  been  designed  to  permit  the  experimenter  to 
define  the  characteristics  of  an  experiment  or  demonstration  including  objects  in  the 
terrain,  flight  plans,  lighting,  aircraft  model  constraints,  auxiliary  displays,  HMD 
symbol  types,  performance  measurement  techniques,  and  a  variety  of  other 
parameters.  For  example.  Figure  5-13  shows  the  pop-up  screen  used  for  defining  the 
nature  of  the  simulated  sunlight  falling  on  the  terrain. 


Figure  5-13.  PRISMS  dialog  page  for  defining  the  sun  simulation. 
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Although  we  attempted  to  create  comprehensive  high-level  interfaces,  we 
recognized  that  the  research  community  is  an  inventive  one,  and  may  create 
entirely  new  methodologies.  To  support  these  advances,  we  provided  a  second  level 
of  user  interaction  with  a  set  of  "tools"  for  building  new  experimental  paradigms, 
metrics,  and  symbol  characteristics  without  a  requirement  for  programming  skills. 

The  result  was  a  rule-based  system  with  which  the  user  specifies  that  if  certain 
conditions  exist,  then  certain  events  will  take  place.  Formalizing  some  rule 
statements  is  easy,  and  for  some  it  is  a  challenging  task.  First  of  all,  it  is  necessary  to 
determine  how  the  knowledge  should  be  described.  In  the  domain  of  expert  systems, 
things  are  typically  described  as  belonging  to  three  categories:  objects,  attributes,  and 
values. 

Objects  can  be  concrete,  such  as  "animals,"  or  conceptual,  such  as  "hygiene." 
Figure  5-14  shows  an  example  (adapted  from  Harmon  &  Sawyer,  1990)  of  an  object 
that  has  attributes.  The  attributes  can  take  on  different  values.  The  values  can  be 
thought  of  as  selections  from  a  list.  Thus,  "body  covering"  might  be  "feathers," 

"fur,"  "scales,"  "skin,"  and  so  forth. 

It  should  be  noted  that  deciding  what  is  an  "object,"  or  an  "attribute"  or  a 
"value"  is  one  of  the  more  intriguing  tasks  in  knowledge  engineering  projects. 
Selecting  the  best  levels  of  abstraction  and  grouping  the  values  within  attributes 
generally  requires  careful  evaluation  and,  sometimes,  repeated  attempts.  For  the 
purposes  of  this  project,  objects  are  usually  (but  not  always)  major  components  such 
as  "the  helicopter"  and  have  attributes  such  as  "airspeed"  and  "altitude"  which  in 
turn  have  values  such  as  "40  knots"  and  "50  feet" 

OBJECTS  ATTRIBUTES  VALUES 


Figure  5-14.  A  diagram  of  an  object  with  attributes  and  values. 


The  objects  and  attributes  already  included  in  the  PRISMS  system  may  be  all 
that  many  users  ever  need.  The  addition  of  new  objects  and  attributes,  however,  is  a 
straightforward  task  and  is  aided  by  the  PRISMS  dialog  system  itself.  A  dialog  page 
summarizing  a  few  of  the  hundreds  of  PRISMS  objects  is  shown  in  Figure  5-15. 
Column  1  presents  the  object  names.  Column  2  the  attribute  name.  Column  3  the 
attribute  type,  and  Column  4  the  current  attribute  value. 
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a  Session  Objects 
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Figure  5-15.  Sample  of  objects  used  in  one  PRISMS  test  session. 

The  dialog  design  has  attempted  to  foresee  a  number  of  experimenter  task 
requirements  and  provide  assistance  through  special,  user-controlled  windows  that 
may  be  displayed  during  the  construction  or  conduct  of  an  experiment.  For  example, 
the  experimenter  may  wish  to  observe  the  changing  values  of  some  particular 
attributes  in  order  to  control  various  events  during  the  simulation  (such  as  the 
sudden  appearance  of  an  enemy  vehicle).  For  such  requirements,  the  PRISMS  dialog 
includes  an  "Attribute  Watch"  window,  as  shown  in  Figure  5-16.  The  experimenter 
can  move  the  window  to  a  corner  of  his  screen  and  add  as  many  attributes  as  desired 
for  monitoring  purposes. 


H  Attribute  Watch 

pgpgf  jin  | 

Object -Attribute  111  || 

Value 

]  distance  from  next  waypoint,  re  suit , 

TUT 

J 

Exposure  Timer. accumulated  time 

'5T27'" 

. . ■ . J 

shot  error,  re  suit 

16  1 

il 

]j 

4 1 . 

. 

v  .-rf:- 

1  ►! 

J  V:..  WM..1 

Figure  5-16.  The  Attribute  Watch  window  for  monitoring  changing  values. 

Once  the  necessary  objects,  attributes,  and  values  have  been  identified, 
inferential  relationships  can  be  stated  in  the  form  of  rules.  A  rule  uses  an  if  clause  to 
identify  one  or  more  conditions  that  must  be  satisfied,  and  a  then  clause  to  indicate 
the  inference.  Both  the  if  and  then  clauses  are  stated  using  a  mix  of  objects, 
attributes,  and  values. 
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Thus,  at  this  second  level  of  user  interaction,  a  unique  logical  dialog  is 
provided  so  that  the  user  can  easily  set  up  an  infinite  variety  of  special  task 
requirements  or  performance  metrics  such  as  "record  elapsed  time  between 
Waypoint  7  crossing  and  designation  of  Target  3  unless  altitude  is  greater  than  200 
ft."  This  development  of  this  dialog  structure  was  a  significant  system  design 
achievement  and  was  critical  to  meeting  the  project  objectives.  The  third  level  of 
interaction  is  the  programming  level,  required  only  for  major  system  changes  that 
could  not  be  anticipated. 

Sample  experiments  were  devised  to  test  the  operation  of  the  higher  level 
dialogs,  and  were  repeatedly  generated  throughout  the  period  of  the  project  so  that 
the  dialogs  would  continue  to  be  enhanced  both  in  power  and  ease  of  use.  Each  time 
a  set  of  sample  experiments  was  devised,  fewer  and  fewer  incremental 
improvements  to  the  dialog  were  needed.  The  specific  techniques  for  creating 
objects,  attributes,  values,  rules  and  HMD  symbology  are  fully  described  in  the 
PRISMS  User  Manual. 

PRISMS  Sound  and  Video  Systems 

PRISMS  includes  a  six-channel  audio  mixer  and  four-channel  audio  amplifier 
for  isolated  communication  between  pilot  and  experimenter,  input  of  helicopter 
and  weapon  sounds,  3D  sounds,  voice  synthesis,  distribution  to  NTSC  output  for 
video,  external  speakers.  Full  matrix  video  switching  and  VGA  to  NTSC/PAL  video 
converter  has  been  provided,  permitting  routing  any  of  three  graphics  channels  to 
any  of  four  output  channels  (experimenter's  monitors,  subject  helmet,  and  NTSC 
video). 

A  3D  sound  generation  has  been  implemented  making  it  possible  to  produce 
sounds  such  that  they  appear  to  the  pilot  to  be  attached  to  a  position  in  space  or  on 
the  ground,  regardless  of  the  pilot's  own  head  movements.  These  sounds  may  be 
used  as  desired  by  future  researchers,  perhaps  evaluating  the  utility  of  a  wingman 
"beep,"  the  positions  and  actions  of  enemy  weapons  systems,  or  spatial  separation  of 
incoming  radio  messages. 

PRISMS  voice  recognition  and  synthesis  systems  have  been  implemented, 
permitting  pilots  to  use  voice  commands  for  changing  symbology  sets  or  any  other 
purposes.  The  experimenter  may  also  use  voice  commands  for  control  of 
experimental  parameters.  The  voice  synthesis  system  can  be  used  with  rule-based 
methods  for  use  in  presenting  voice  warnings  such  as  used  with  the  APR-39 
warning  system,  or  for  other  cockpit  alerts. 
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Section  6:  Symbology  Evaluations  with  PRISMS; 
Experimental  Method  and  Metrics 


Introduction 

Having  conducted  the  information  requirements  analyses  and  developed  the 
PRISMS  simulator,  we  were  in  a  position  to  begin  conducting  hundreds  of  studies  of 
new  HMD  symbology.  Unfortunately,  only  a  few  of  these  could  be  undertaken 
within  the  time  and  resources  available  and  we  were  faced  with  restricting  our 
research  effort  to  a  small  fraction  of  that  possible.  Thus,  we  attempted  to  identify  the 
new  symbols  that  were  likely  to  provide  the  highest  payoff  based  on  our  knowledge 
of  the  operational  requirements  and  current  shortcomings.  In  doing  so,  we 
consulted  our  analytical  data  as  well  as  our  subject  matter  experts.  The  most  impor¬ 
tant  symbols  were  included  as  independent  variables  in  our  experiment,  and  the 
runners-up  were  evaluated  by  flight  demonstrations  on  PRISMS,  followed  by  inter¬ 
view  sessions. 

The  Attack  Mission  Experiment 

In  structuring  these  data-gathering  techniques,  we  attempted  to  make  the  best 
trade-off  between  experimental  control  and  realistic  mission  relevance,  pretesting 
our  methods  with  expert  pilots.  We  also  selected  research  topics  to  demonstrate  the 
breadth  of  PRISMS'  capabilities  for  symbol  development,  performance  measure¬ 
ment,  and  flexibility  in  adding  and  modifying  study  session  features  and 
characteristics. 

This  section  of  the  report  describes  the  experimental  and  demonstration 
sessions  in  detail.  The  experiment  is  discussed  in  terms  of  the  necessary  prepara¬ 
tions,  participants,  apparatus,  procedures,  experimental  design,  and  the  results  of  13 
experimental  comparisons. 

Method 

Participants 

We  initially  made  arrangements  for  an  August  1998  visit  to  interview  Apache 
Instructor  Pilots  at  Fort  Rucker.  Unfortunately,  because  of  subsequent  computer  failures 
leading  to  unavoidable  slips  in  the  Air  Warrior  test  schedule,  we  were  notified  that  no 
pilots  could  be  made  available  for  our  work.  We  then  contacted  Lt.  Col  Michael  Jensen, 
Commander,  1st  Battalion  (Attack),  211th  Aviation  Regiment,  of  West  Jordan,  Utah.  Lt.Col 
Jensen  graciously  arranged  for  our  visit  there  in  late  September,  1998,  where  14  Apache 
pilots  participated  in  a  series  of  simulated  flights  to  gather  data  on  the  new  HMD  symbol¬ 
ogy  and  symbology  moding  strategies. 

Their  average  number  of  helicopter  flight  hours  was  2,368  (range  700-7800)  and 
their  average  number  of  hours  in  the  AH-64  was  1,090  (range  380-3000).  The  pilots 
were  advised  that  the  purpose  of  the  study  was  to  evaluate  potential  new  HMD 
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symbols  in  the  light  of  operational  requirements  and  that  they  would  fly  the 
PRISMS  simulator  with  and  without  the  new  symbols  in  order  to  gather  data  and 
facilitate  discussions  about  symbol  utility  and  rules  for  effective  presentation  of  the 
symbol  in  the  HMD.  They  were  also  told  that  individual  sources  of  opinions  would 
remain  confidential. 

Apparatus  and  Materials 

Terrain  areas  selected.  The  terrain  selected  for  the  experiment  was  located  in 
south-central  New  Mexico  in  an  area  with  mixed  vertical  development  ranging 
from  flat  desert  to  low  hills  to  mountains  about  3,000  feet  above  the  desert  floor. 

The  specific  area  used  for  the  familiarization  flight  is  found  on  the  USGS  1:100,000- 
scale  map  entitled  “Tularosa,”  showing  Rhodes  Canyon  passing  through  the  San 
Andres  Mountains.  A  portion  of  this  topographic  map  is  shown  in  Figure  6-1. 


The  route  through  Rhodes  Canyon  was  chosen  for  familiarization  because  its 
dramatic  and  easily  distinguishable  features  as  well  as  the  requirement  for  frequent 
heading  and  altitude  changes  to  closely  follow  the  bottom  of  the  canyon. 

The  area  selected  for  the  experiment  was  just  north  of  the  familiarization  area 
and  is  shown  on  the  USGS  l:24,000-scale  7.5  x  7.5  minute  quadrangle  map  entitled 
"Fairview  Mountain."  The  terrain  in  the  southeast  portion  of  this  map  sheet 
provided  nearly  ideal  landforms  for  a  helicopter  attack  mission,  including  cover  and 
concealment  along  the  route,  good  battle  positions  behind  a  ridgeline  masking  the 
BPs  from  the  enemy  armor  column,  and  a  large  hill  to  the  rear  of  the  BPs  so  that  the 
aircraft  would  not  be  "sky-lined"  during  the  bob-up  maneuver.  A  copy  of  a  portion 
of  this  map  is  presented  in  Figure  6-2. 

Procedure 

Familiarization  flight.  The  pilots  were  told  that  they  would  have  an  opportu¬ 
nity  to  get  the  feel  of  the  PRISMS  handling  characteristics  by  doing  a  hover  and 
landing  on  a  pad,  and  then  flying  down  a  canyon.  They  were  also  shown  a  drawing 
of  the  virtual  waypoint  symbol  as  it  would  appear  later  in  the  flight,  and  a  screen 
image  of  a  tank  that  would  appear  in  the  terrain  as  a  "target  of  opportunity"  to  be 
fired  on.  Subjects  then  donned  the  helmet,  adjusted  the  inter-pupilary  distance  and 
individual  eye  focus  and  sat  motionless  for  a  few  seconds  to  complete  head-tracker 
calibration.  The  three  controls  buttons  on  the  cyclic  grip  (trigger,  symbol  mode 
switch,  and  map-stepping  switch)  were  indicated  to  the  pilot,  and  the  pilot  was 
requested  to  cycle  through  the  four  symbol  modes  as  well  as  the  various  head-down 
map  views.  Pilots  were  instructed  to  fly  at  about  100  feet  and  60  knots.  During  the 
familiarization  flight,  which  lasted  about  ten  minutes,  the  experimenter  pointed  out 
how  the  command  heading  cue  could  be  used  to  find  waypoints,  and  later,  how  the 
white  "lollipop"  waypoint  symbol  could  be  used  to  identify  waypoint  positions  in 
the  terrain.  It  was  also  explained  that  if  the  next  waypoint  position  was  obscured  by 
terrain  between  it  and  the  aircraft  position,  it  would  be  shown  by  a  dashed  red 
outline.  Two  "targets  of  opportunity"  (tanks)  were  pointed  out  by  the  experimenter, 
and  fired  upon  by  the  pilots. 

Experimental  tasks  -  unaided.  The  pilots  were  instructed  to  examine  a  1:24,000- 
scale  paper  map  of  the  area  of  operations  complete  with  a  attack  mission  plan  over¬ 
lay  and  drawing  of  the  EA.  They  were  reminded  of  the  head-down  map  visible  in 
the  helmet  and  told  that  it  would  display  a  series  of  overlapping  portions  of  the 
route  shown  in  the  paper  map.  They  were  told  that  they  were  to  overfly  all  of  the 
waypoints  and  that  the  map  and  the  command  heading  caret  in  the  magnetic 
heading  scale  would  be  their  only  navigation  aids.  They  were  further  instructed  to 
maintain  100  feet  AGL  altitude  and  a  60  knot  airspeed,  to  detect  and  fire  on  targets  of 
opportunity,  and  to  land  at  the  Flolding  Area  (HA)  at  Waypoint  5. 

After  landing  at  the  HA,  the  pilots  were  instructed  to  examine  the  HMD  map 
segment  showing  the  path  to  the  BPs  and  that  they  should  fly  to  and  land  at  the 
center  BP,  where  the  Command  Heading  Marker  would  lead  them.  They  were 
instructed  to  proceed  at  approximately  40  knots,  to  maintain  masking  from  the 
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Figure  6-2.  A  part  of  the  topographic  map  of  the  experimental  area. 
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enemy  positions  to  the  west,  and  to  land  at  the  BP  as  accurately  as  possible.  After 
landing  at  the  BP,  the  pilots  were  instructed  to  look  at  the  HMD  map  segment 
showing  the  EA  in  its  entirety  and  to  observe  the  three  firing  sectors.  They  were 
told  that  upon  bob-up  that  they  would  see  a  column  of  enemy  tanks  (16  tanks  were 
visible)  and  that  they  were  to  fire  on  only  two  tanks:  the  one  closest  to  the  left 
boundary  of  the  center  sector  and  the  one  closest  to  the  right  boundary  of  the  center 
sector.  They  were  urged  to  fire  within  15  seconds  and  then  bob  down  to  the  BP. 

Experimental  tasks  -  aided.  In  all  cases,  the  aided  tasks  were  performed  after  the 
unaided  tasks.  After  a  rest  period  of  about  two  minutes,  the  pilots  were  instructed 
that  the  next  condition  would  be  the  same  as  the  previous  one  except  that  they 
would  be  aided  by  the  virtual  waypoint  markers,  or  “lollipops”  along  the  route. 

They  were  also  told  that  when  they  closely  approached  the  waypoints,  they  could  see 
a  10-foot  hemisphere  marking  the  exact  spot  on  the  ground,  as  shown  in  Figure  6-3. 
These  hemispheres  could  be  used  to  aid  accurate  landings  at  the  HA  and  the  BP, 
especially  if  the  aircraft  had  overflown  the  waypoint  and  the  waypoint  marker  had 
jumped  to  the  next  waypoint.  The  rest  of  the  instructions  were  identical  to  those  of 
the  unaided  condition. 


Figure  6-3.  Examples  of  the  “lollipop”  and  “igloo”  waypoint  markers. 

After  landing  at  the  HA  in  the  aided  condition,  the  pilots  were  instructed  that 
in  addition  to  the  BP  marker,  they  would  see  the  EA  marked  out  with  red  lines  in 
the  terrain  and  that  if  the  aircraft  was  masked  from  the  EA,  the  red  lines  would 
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appear  as  dashed  lines.  Figure  6-4  shows  the  appearance  of  the  EA  from  the  south, 
near  the  HA.  In  this  example,  the  aircraft  is  at  an  altitude  of  489  feet  above  ground 
level,  and  clearly  exposed  to  the  enemy  forces. 


Figure  6-4.  View  of  the  virtual  EA  marker  symbol  from  the  south. 

Upon  landing  at  the  BP  in  the  aided  condition,  the  pilots  received  the  same 
instructions  as  in  the  unaided  condition,  except  they  were  told  that  they  would  be 
able  to  see  the  EA  in  the  terrain  and  should  shoot  the  two  tanks  nearest  the  lines 
marking  the  left  and  right  boundaries  of  the  center  sector. 

Figures  6-5  and  6-6  show  the  pilots  view  of  the  EA  symbology  as  it  is  seen  from 
the  BP.  Figure  6-5  shows  an  example  of  the  dotted  lines  of  the  symbol  becoming 
solid  as  the  aircraft  rises  from  a  masked  position  behind  the  ridgeline.  Figure  6-6 
shows  the  appearance  of  the  assigned  central  section  of  the  EA  viewed  from  above 
the  BP. 

Experimental  Design 

Independent  variables.  The  only  independent  variable  was  the  presence  or 
absence  of  the  earth-fixed  HMD  symbology  marking  the  waypoints  and  the  EA 
boundaries.  To  control  for  the  very  large  range  of  individual  differences  in 
performance,  a  repeated  measures  design  was  selected  in  which  each  subject  flew 
both  the  unaided  and  then  the  aided  conditions.  During  the  first  flight  through  the 
mission  area,  the  pilots  received  no  feedback  from  the  experimenters  regarding  the 
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Figure  6-6.  Direct  view  of  the  central  section  of  the  EA  from  the  BP 
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accuracy  of  their  waypoint  crossings,  landings,  shots  at  the  EA,  or  on  any  of  the  other 
performance  metrics.  During  the  second  flight,  the  earth  fixed  symbols  provided  the 
pilots  with  direct  feedback  on  their  performance  accuracy. 

Dependent  variables.  The  earth-fixed  symbology  was  expected  to  greatly  reduce 
pilot  workload  in  the  navigation  tasks  and  increase  situation  awareness.  Because 
performance  measurements  are  not  always  clearly  related  to  these  improvements,  a 
number  of  different  metrics  were  employed  in  attempting  to  identify  enhanced 
performance.  These  included  the  accuracy  of  waypoint  passage  (feet),  the  number  of 
targets  of  opportunity  detected  (number  fired  upon),  the  landing  distances  from  the 
HA  and  BP  (feet),  the  accuracy  of  altitude  maintenance  (RMSE  feet),  the  accuracy  of 
airspeed  maintenance  (RMSE  feet),  the  total  exposure  time  to  the  EA  enroute 
(seconds),  the  total  number  of  exposure  events,  the  total  exposure  time  during  the 
bob-up  (seconds),  the  number  of  inadvertent  ground  contacts,  the  accuracy  of  target 
selection  in  the  EA  (feet),  and  the  time  (seconds)  that  the  head  angle  fell  within  each 
of  nine  20-degree  sectors  in  the  frontal  180°  field  of  regard  for  each  flight  leg. 

Subjective  data.  In  addition  to  the  performance  measures  described  above,  after 
the  aided  condition  was  completed,  the  pilots  were  requested  to  estimate  the  work¬ 
load  reduction  percentage  resulting  from  the  presence  of  the  waypoint  markers  and 
also  from  the  presence  of  the  EA  sector  lines.  As  an  extension  to  these  estimates, 
they  were  queried  regarding  the  perceived  utility  of  the  earth-fixed  symbols,  their 
suggestions  for  improving  them,  and  their  recommendations  for  the  conditions 
under  which  these  symbols  should  appear  and  disappear  from  the  symbology  set. 


Results 

Data  Analysis 

Accuracy  of  waypoint  passage.  The  accuracy  of  waypoint  passage  (the  sum  of 
passage  distances  for  Waypoints  1  to  4,  the  HA,  and  the  BP),  was  significantly  better 
for  the  aided  group,  t(13)  =  7.276,  p<.001,  with  a  mean  distance  of  878  feet  (SD  =  373) 
vs.  287  feet  (SD  =  215).  Every  pilot  in  the  study  was  considerably  more  accurate  with 
the  earth-fixed  aids  to  waypoint  recognition  than  without  them.  The  relative  differ¬ 
ence  between  unaided  and  aided  conditions  was  consistent  across  all  six  waypoints. 


Accuracy  of  waypoint  passage 
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Figure  6-7.  Waypoint  passage  results. 
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Landing  distances  from  the  HA.  The  distances  between  the  designated  position 
of  the  HA  and  the  actual  landing  positions  of  the  aircraft  differed  significantly,  t(12) 

=  4.679,  pc. 001;  the  mean  distance  in  the  unaided  condition  was  1,130  feet  (SD  =  588) 
and  the  mean  distance  in  the  aided  condition  was  262  feet  (SD  =  481).  All  of  the 
pilots  but  one  were  closer  to  the  HA  with  the  earth-fixed  waypoint  markers.  Had 
that  pilot's  score  not  been  included  in  the  group,  the  mean  distance  from  the  HA  for 
the  aided  condition  would  have  been  halved  (131  feet). 
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Figure  6-8.  HA  landing  distance  results. 

Landing  distances  from  the  BP.  The  distances  between  the  designated  position 
of  the  BP  and  the  actual  landing  positions  of  the  aircraft  differed  significantly,  t(ll)  = 
5.458,  pc.001,  in  a  manner  very  similar  to  that  observed  with  the  HA  landing 
distances.  The  mean  distance  in  the  unaided  condition  was  1,111  feet  (SD  =  529)  and 
the  mean  distance  in  the  aided  condition  was  256  feet  (SD  =  406).  All  of  the  pilots 
landed  closer  to  the  BP  with  the  waypoint  markers  than  without  them.  Again, 
however,  one  of  the  pilots  made  a  large  landing  error  (1500  feet)  even  with  the  way- 
point  marker.  Had  his  score  not  been  included  in  the  group,  the  mean  distance 
from  the  HA  for  the  aided  condition  would  have  been  much  smaller  (143  feet). 
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Figure  6-9.  BP  landing  distance  results. 
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Mean  exposure  time  to  the  EA  enroute.  In  both  unaided  and  aided  conditions, 
pilots  were  advised  to  maintain  masking  behind  landforms  to  prevent  their  detec¬ 
tion  by  enemy  forces  in  the  EA.  The  unaided  group  accumulated  an  average  of  28.7 
seconds  of  exposure  time  (SD  =  27.0),  and  the  aided  group  accumulated  an  average  of 
only  15.8  seconds  (SD  =  22.2).  This  difference  was  significant;  t(13)  =  2.234,  p<.05. 


Mean  exposure  time  to  the  EA  enroute 
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Figure  6-10.  Exposure  time  results. 

Mean  number  of  exposure  events.  The  unaided  group  exposure  events  ranged 
from  zero  (one  pilot)  to  four  with  a  mean  of  1.93  events  (SD  =  .997).  The  aided 
group  exposure  events  ranged  from  zero  (six  pilots)  to  three  with  a  mean  of  1.21 
events  (SD  =  1.188).  This  difference  was  also  significant;  t(13)  =  2.687,  p<.025. 


Mean  number  of  exposure  events 
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Figure  6-11.  Exposure  event  results. 

Total  exposure  time  during  the  bob-up.  The  exposure  time  during  the  bob-up 
maneuver  was  collected  primarily  to  indicate  whether  or  not  the  pilots  were  able  to 
comply  with  the  experiment  instructions  to  complete  the  maneuver  rapidly.  The 
exposure  time  prior  to  firing  the  second  shot  was  22.7  seconds  (SD  =  12.3)  for  the 
unaided  condition  and  20.5  seconds  (SD  =  13.0)  for  the  aided  condition.  The  expo¬ 
sure  times  from  first  exposure  to  remasking  after  the  bob-down  were  42.3  seconds 
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(SD  =19.7)  for  the  unaided  condition  and  39  seconds  (SD  =18.6)  for  the  aided  condi¬ 
tion.  Neither  of  these  differences  were  statistically  significant. 

Accuracy  of  altitude  maintenance.  The  mean  RMSE  around  the  100  foot  AGL 
altitude  was  112  feet  (SD  =48.1)  for  the  unaided  condition  and  93  feet  (SD  =  42.7)  for 
the  aided  condition.  This  difference  was  not  statistically  significant. 

Accuracy  of  airspeed  maintenance.  The  mean  RMSE  about  the  60-knot  airspeed 
was  16  knots  (SD  =  7.1)  for  the  unaided  condition  and  22  knots  (SD  =6.5)  for  the 
aided  condition.  This  difference  was  not  statistically  significant. 

Number  of  targets  of  opportunity  detected.  The  average  number  of  targets  of 
opportunity  hit,  out  of  a  possible  4  was  1.6  for  the  unaided  condition  (SD  =  .73)  and 
1.7  for  the  aided  condition  (SD  =  1.43).  This  difference  was  not  statistically  signifi¬ 
cant. 


Accuracy  of  sector  identification  in  the  EA.  The  utility  of  the  earth-fixed  EA 
symbology  was  particularly  dramatic.  The  average  error  (as  a  sum  of  the  two  shots) 
was  1666  feet  (SD  =  869)  in  the  unaided  condition  and  14  feet  (SD  =  54)  in  the  aided 
condition.  Not  surprisingly  this  difference  was  statistically  significant  t(13)  =  7.147, 
pc.OOl.  In  the  unaided  condition,  only  3  of  28  shots  were  directed  at  the  correct 
points.  In  the  aided  condition,  27  of  28  shots  were  fired  at  the  tanks  on  the  sector 
boundaries;  there  was  only  one  stray  shot  that  was  directed  at  a  tank  201  feet  away 
from  the  correct  one. 


Accuracy  of  sector  identification  in  the  EA 
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Figure  6-12.  Firing  sector  identification  results. 

It  should  be  emphasized  that  the  errors  in  the  unaided  condition  resulted  both 
from  inaccurately  selecting  the  battle  position,  as  well  as  inaccuracies  in  map  inter¬ 
pretation  and  terrain  analysis  in  identifying  the  sector  boundaries.  In  the  aided 
condition,  the  landing  distances  were  several  hundred  feet  closer  to  the  BP,  on  the 
average,  than  in  the  unaided  condition.  Nevertheless,  BP  errors  do  not  explain  the 
great  differences  in  performance  between  the  aided  and  unaided  conditions.  The 
comparison  of  the  shot  accuracy  and  the  BP  accuracy  for  the  unaided  and  aided 
conditions  is  facilitated  by  Figures  6-13  and  6-14. 
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Unaided  condition.  The  shots  taken  by 
the  14  Apache  pilots  in  attempting  to 
identify  the  tanks  on  the  boundaries 
of  the  center  fire  sector.  The  +  signs 
show  the  position  from  which  the  shots 
were  fired  (the  pilots  identification  of 
the  BP  position).  The  asterisks  indicate 
four  cases  in  which  BP  selection  error 
was  exceedingly  large  and  the  aircraft 
was  artificially  returned  to  the  correct 
BP  by  the  experimenter. 
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Aided  condition.  The  shots  taken  by  thel4 
Apache  pilots  in  attempting  to  identify  the 
tanks  on  the  boundaries  of  the  center  fire 
sector.  The  +  signs  show  the  position  from 
which  the  shots  were  fired  (the  pilots 
identification  of  the  BP  position).  The 
asterisk  indicates  one  case  in  which  BP 
selection  error  was  exceedingly  large  and 
the  aircraft  was  artificially  returned  to  the 
correct  BP  by  the  experimenter.  The  # 
indicates  one  case  in  which  the  firing 
position  could  not  be  exactly  identified.  The 
E  shows  the  single  error. 


Figure  6-14.  Shots  fired  in  the  EA  in  the  aided  condition. 
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Figure  6-13  shows  that  no  pilot  in  the  unaided  group  was  able  to  successfully 
identify  the  assigned  fire  sector.  Regardless  of  the  firing  position,  the  extent  of  the 
assigned  firing  sector  shown  on  the  map  was  vastly  over-  and  under-estimated 
when  viewing  the  terrain.  Although  their  BP  selection  errors  were  often  large, 
elimination  of  these  errors  did  not  result  in  correct  firing  sector  identification,  as 
demonstrated  by  the  scores  of  the  four  pilots  shown  at  the  bottom  of  Figure  6-13. 

During  conduct  of  the  unaided  condition,  these  four  pilots  missed  the 
command  heading  cue  shift  and  overflew  the  BP  by  large  distances.  In  these  cases, 
the  experimenter  stopped  the  aircraft,  scored  a  1500  foot  error,  and  returned  the 
aircraft  to  the  BP.  As  shown  at  the  bottom  of  Figure  6-13,  however,  these  pilots 
were,  like  the  other  ten  pilots,  unable  to  accurately  identify  the  tanks  marking  the 
fire  sector  boundaries.  As  further  evidence  that  the  BP  selection  error  did  not 
account  for  the  EA  shot  errors,  four  other  pilots  in  the  unaided  group  landed  fairly 
closed  to  the  BP  but  were  unable  to  accurately  identify  the  firing  sector. 

In  contrast,  Figure  6-14,  in  the  aided  condition,  shows  that  nearly  all  (27  of  28)  of 
the  shots  were  correct,  even  though  the  pilots  had  misidentified  the  BP  position  by 
an  average  of  226  feet.  The  reason  for  the  single  shot  error  is  unclear.  It  may  have 
been  that  the  correct  target  was  momentarily  obscured  behind  the  canopy  rail  or 
simply  that  a  transitory  head  alignment  error  shifted  the  reticle  toward  the  adjacent 
tank. 

Workload  reduction  estimates.  Although  there  many  techniques  have  been 
used  to  estimate  workload  and  workload  reduction,  most  of  these  require  a  substan¬ 
tial  investment  of  time.  In  attempting  to  gather  as  much  flight  and  knowledge  data 
as  possible  in  the  90  minutes  available  with  each  pilot,  we  chose  a  very  simple  metric 
for  workload  reduction  estimates.  We  simply  asked  the  pilots,  given  the  flights  with 
and  without  the  ground-fixed  symbols  in  PRISMS,  how  much  their  workload  would 
be  reduced  for  similar  operations  in  the  aircraft.  The  responses  were  extraordinarily 
favorable  for  these  symbols.  The  mean  workload  reduction  attributable  to  the  way- 
point  symbols  was  estimated  at  55%  (SD  =  25.4)  The  mean  workload  reduction 
attributable  to  the  EA  symbols  was  estimated  at  69%  (SD  =  24.3).  These  are  certainly 
attention-getting  votes  of  confidence  for  these  new  symbol  types.  Additional  inter¬ 
view  information  supplementing  these  figures  is  described  later  in  this  report. 

Head  angle  measurements.  One  of  the  objectives  of  providing  pilots  with 
earth-fixed  symbols  is  to  increase  situation  awareness.  Earth-fixed  symbols  could  do 
this  through  indicating  the  positions  of  important  elements  in  the  head  up  view 
and  also  through  reducing  workload  such  that  the  pilot  would  have  more  time 
available  to  turn  his  head  and  scan  the  terrain  surrounding  him.  In  order  to  deter¬ 
mine  whether  such  an  effect  was  immediately  observable,  timers  were  used  to 
measure  the  time  in  seconds  that  the  head  angle  fell  within  each  of  nine  20-degree 
sectors  in  the  frontal  180°  field  of  regard,  as  shown  in  Figure  6-15.  The  head  angle 
metrics  were  summed  across  the  first  four  flight  legs,  the  legs  between  Waypoint  1 
and  the  HA,  in  which  the  pilots  had  been  instructed  to  search  for  targets  of  opportu¬ 
nity.  The  field  of  view  of  the  helmet  was  60°  x  47°,  so  the  pilot's  eye  movements 
would  allow  him  to  view  up  to  an  additional  30  degrees  to  the  side  beyond  the  head 


6-14 


angle  indicated  by  the  timer.  In  the  unaided  condition,  pilots  head  angles  were 
limited  to  the  center  sector  during  51%  of  the  time  (SD  =  15.5).  Surprisingly,  in  the 
aided  condition,  pilots  head  angles  were  limited  to  the  center  sector  during  67%  of 
the  time  (SD  =  12.3).  This  difference  is  statistically  significant;  t(13)  =  4.99,  p<.001). 
The  distribution  of  times  across  the  nine  sectors  is  shown  in  Figure  6-16. 


Number  of  inadvertent  ground  contacts.  The  measurement  of  the  number  of 
inadvertent  ground  contacts  was  added  almost  as  an  afterthought.  This  was  a  fortu¬ 
nate  inclusion  because  it  revealed  some  dramatic  differences  between  the  aided  and 
unaided  conditions.  Only  three  of  the  pilots  in  the  unaided  condition  struck  the 
ground  during  the  mission,  and  none  struck  the  ground  more  than  once  (Mean  = 
0.21,  SD  =  0.43).  In  marked  contrast,  in  the  aided  condition  every  single  pilot  struck 
the  ground  at  least  once,  and  seven  of  them  struck  the  ground  more  than  once 
(Mean  =1.71,  SD  =  1.07)  for  a  total  of  24  ground  strikes.  This  difference  is  statistically 
significant:  t(13)  =  -4.364  pc.OOl,  and  is  also  of  great  practical  significance.  How  is  it 
that  the  waypoint  symbols,  despite  their  demonstrated  superiority  in  waypoint 
passing  distance,  HA  and  BP  landing  distances,  and  estimated  workload  reduction, 
can  lead  to  eight  times  as  many  inadvertent  ground  contacts?  This  question  will  be 
addressed  in  the  Discussion  section. 

Discussion  of  Experiment  Results 

Repeated  Measures  Design 

There  are  two  basic  methods  for  designing  an  experiment  such  as  this  one. 

First,  it  would  be  possible  to  allocate  test  subjects  (pilots)  to  one  of  two  groups:  the 
unaided  group  and  the  aided  group,  perhaps  attempting  to  somehow  match  the 
groups  for  experience,  skill,  or  aptitude.  Because  of  the  very  large  range  of  individ¬ 
ual  differences  in  pilot  capabilities,  a  fairly  large  number  of  pilots  would  be  needed 
in  each  group;  perhaps  12  per  group,  at  a  minimum.  Unfortunately,  it  is  nearly 
impossible  to  obtain  access  to  a  group  of  24  or  more  Apache  pilots,  simply  because  of 
the  existing  demands  on  their  time. 

The  second  method  for  designing  an  experiment  to  compare  two  conditions  is 
the  "repeated  measures"  approach,  letting  each  individual  serve  as  his  or  her  own 
control  group  by  participating  in  both  conditions.  Typically,  to  control  for  practice 
effects  or  other  potential  effects  of  performing  the  tasks  in  a  certain  order,  half  of  the 
subjects  perform  the  two  conditions  in  one  order  and  the  other  half  perform  them 
in  the  opposite  order. 

This  dual-order  approach  was  impossible  in  the  present  experiment  because  the 
aided  condition  could  not  be  shown  before  the  unaided  condition.  That  is,  the  way- 
points  and  EA  markers  showed  the  correct  positions  so  that  the  pilots  would  still 
remember  where  they  were  (approximately)  when  they  performed  the  unaided 
condition.  Thus,  all  of  the  pilots  in  our  experiment  flew  the  unaided  condition 
followed  by  the  aided  condition  and  it  is  not  impossible  that  there  was  some 
learning  during  the  first  flight  that  might  have  aided  in  the  performance  of  the 
second.  However,  during  the  first  flight,  the  experimenters  were  careful  not  to 
provide  any  feedback  to  the  pilot  regarding  the  accuracy  of  their  waypoint  crossings, 
landings,  shots  at  the  sector  boundaries  in  the  EA,  or  on  any  other  performance 
metric.  So,  any  learning  that  took  place  would  not  have  directly  influenced  the 
scores  on  these  performance  measures. 
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As  a  check  on  potential  learning  effects,  we  considered  having  a  few  pilots  fly 
the  unaided  condition  twice,  to  determine  whether  the  metric  scores  improved  on 
the  second  unaided  flight.  However,  after  having  observed  the  very  large  inter¬ 
subject  and  intra-subject  performance  variability,  we  decided  that  little  would  be 
gained  from  such  a  method  unless  many  pilots  were  included.  Unfortunately,  such 
an  approach  would  have  reduced  the  number  of  pilots  available  to  fly  in  the  experi¬ 
ment  itself.  In  any  case,  from  the  dramatic  differences  in  performance  and  from  the 
high  praise  for  the  symbols  from  the  pilots,  it  seems  clear  that  the  experimental 
results  are  almost  entirely  based  on  the  presence  or  absence  of  the  earth-fixed 
symbols  and  not  on  practice  effects. 

Accuracy  of  Position  Finding 

Three  metrics  were  used  to  determine  the  pilot's  ability  to  arrive  at  designated 
points  with  and  without  the  new  earth-fixed  waypoint  markers:  accuracy  of 
waypoint  passage,  landing  distance  from  the  HA,  and  landing  distance  from  the  BP. 
The  pilots'  performance  on  these  metrics  showed  that  with  the  virtual  markers, 
errors  were  only  33%,  23%,  and  23%  as  great,  respectively,  as  those  observed  without 
these  earth-fixed  markers. 

The  magnitude  of  these  improvements  was  somewhat  surprising  since  the 
command  heading  caret  alone  could  be  used  (and  was  in  some  instances)  to  accu¬ 
rately  reach  the  waypoint.  In  order  to  do  so,  however,  the  caret  had  to  be  kept 
centered  continuously.  If  error  distance  was  allowed  to  begin  increasing  rather  than 
decreasing  when  the  aircraft  was  already  close  to  the  waypoint  (within  500  feet),  the 
system  would  automatically  switch  the  command  heading  symbol  to  the  next  way- 
point  in  the  logical  assumption  that  the  waypoint  had  been  passed.  Because  the  pilot 
did  not  know  the  exact  distance  to  the  waypoints,  it  was  thus  imperative  to  keep  the 
caret  centered.  Not  only  is  keeping  the  caret  centered  a  more  demanding  task  than 
simply  flying  toward  a  waypoint  symbol  visible  in  the  terrain,  but  enforced  straight- 
line  flight  does  not  permit  the  pilot  to  take  advantage  of  terrain  for  cover  and 
concealment  by  following  low  areas  that  are  off  the  most  direct  path  to  the  next 
waypoint. 

Although  some  pilots  are  quite  adept  at  map  interpretation  and  terrain  analy¬ 
sis,  permitting  reasonably  accurate  estimates  of  current  aircraft  position,  many  have 
become  accustomed  to  depending  upon  the  CPG  to  use  the  Doppler  and  the  paper 
map  to  periodically  determine  present  position  and  distance  to  the  next  waypoint 
and  to  verbally  relay  them  to  the  pilot. 

In  the  unaided  condition,  the  pilot  was  tasked  with  estimating  waypoint 
distance  (based  on  dead  reckoning  and  map-terrain  correlation)  and  slowing  appro¬ 
priately  for  landings.  When  the  command  heading  caret  moved  rapidly  to  a  new 
direction,  the  pilot  knew  that  the  waypoint,  HA,  or  BP  had  been  reached,  but  if  the 
aircraft  were  still  traveling  at  60  knots,  it  would  be  certain  to  overfly  the  position 
before  landing.  In  contrast,  the  waypoint  virtual  markers  permitted  the  pilots  to 
slow  prior  to  landing,  and  furthermore,  to  have  a  clear  indication  of  the  specific 
position  designated  for  the  landing. 
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Exposure  Metrics 

Two  metrics  were  employed  to  determine  the  pilot's  ability  to  traverse  the  ter¬ 
rain  between  the  HA  and  the  BP  without  intervisibility  between  the  aircraft  and  the 
enemy  armor  unit  to  the  west:  total  exposure  time  to  the  EA  and  total  number  of 
exposure  events.  The  condition  aided  by  the  virtual  EA  markers  produced  only  55% 
as  much  exposure  time,  and  63%  as  many  exposure  events. 

The  pilots  were  instructed  to  fly  at  about  40  knots,  but  no  flight  altitude  was 
suggested  other  than  that  they  were  to  "maintain  masking"  from  the  enemy  posi¬ 
tions.  In  fact,  an  altitude  of  approximately  50  feet  was  the  upper  limit  for  maintain¬ 
ing  masking.  The  ridgeline  between  the  enemy  tanks  and  the  path  to  the  BP  was  of 
sufficient  altitude  to  provide  continuous  masking. 

It  is  likely  that  the  superior  results  for  the  aided  condition  result  from  the  cues 
provided  by  the  dotted  red  lines  of  the  EA  symbol.  The  EA  lines  are  depicted  as  dots 
when  the  EA  is  obscured  by  the  masking  terrain.  Thus,  if  the  dotted  red  lines  moved 
near  the  top  of  the  masking  terrain  it  would  be  evident  that  masking  was  about  to  be 
lost.  And,  if  the  dotted  lines  became  solid,  this  event  would  provided  an  unmistak¬ 
able  warning  of  exposure  to  the  enemy  forces.  Although  enemy  tanks  could  be  seen 
by  the  pilot  during  moments  of  exposure,  they  would  have  been  a  much  less  salient 
cue  than  the  bright  solid  red  lines  of  the  EA. 

Secondary  Task  Metrics 

Three  additional  metrics  were  employed  to  attempt  to  observe  indirect  benefits 
of  the  virtual  waypoint  markers:  maintaining  designated  altitude  and  airspeed,  and 
searching  for  targets  of  opportunity.  We  had  expected  that  the  virtual  waypoint 
markers  would  greatly  reduce  workload,  as  indeed  the  subjective  reports  dramati¬ 
cally  proved.  It  was  surmised  that  reduced  workload  in  the  navigation  task  might  be 
reflected  in  the  secondary  tasks,  but  the  performances  of  these  tasks,  did  not  signifi¬ 
cantly  differ  between  the  aided  and  unaided  conditions. 

Although  it  seemed  probable  that  reduction  of  overall  workload  resulting  from 
use  of  the  earth-fixed  symbology  might  be  reflected  in  the  pilots'  ability  to  devote 
more  cognitive  resources  to  these  secondary  tasks,  there  is  no  evidence  that  this  was 
the  case.  The  lack  of  a  significant  difference  in  the  altitude  and  airspeed  metrics  does 
not  indicate  that  workload  was  not  reduced  by  the  new  symbols;  only  that  any  newly 
available  resources  were  not  allocated  to  these  particular  tasks.  Such  resources  might 
have  been  devoted  to  other  flight  and  tactical  tasks,  such  as  maintaining  aircraft  atti¬ 
tude  or  map-terrain  correlation  activities. 

For  the  target  search  task,  like  the  altitude  and  airspeed  maintenance  tasks,  it 
seemed  reasonable  that  a  reduction  in  overall  workload  from  use  of  the  ground- 
fixed  waypoint  markers  might  be  reflected  in  more  time  available  to  the  pilots  for 
turning  their  heads  to  search  the  terrain  for  targets.  Because  most  of  the  targets  were 
placed  well  off  the  flight  path  and  behind  landforms,  rather  large  (but  not  uncom¬ 
fortable)  head  movements  were  required  for  their  detection.  In  general,  however,  it 
was  revealed  that  the  pilots  spent  most  of  the  time  with  their  heads  addressing  the 
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center  20°  sector  of  their  potential  field  of  regard,  regardless  of  the  presence  or 
absence  of  waypoint  marker  symbols. 

Head  Scanning  and  Ground  Contacts 

Despite  the  pilots'  overwhelming  preference  for  the  virtual  waypoint  symbols 
and  the  very  large  performance  enhancements  attributable  to  them,  two  of  the 
outcomes  of  their  presence  were  disturbing.  First,  the  reduced  head  scanning  behav¬ 
ior  and  second,  the  increased  frequency  of  inadvertent  ground  contacts  observed  in 
the  aided  condition  seem  to  indicate  that  the  new  symbology  was  unduly  distracting 
the  pilots  from  their  normal  head  movement  scanning  and  their  HMD  symbology 
scanning.  The  head  angle  measurements  showed  that  the  pilots  in  the  aided  condi¬ 
tion  spent  31%  more  time  looking  roughly  straight  ahead  than  when  they  were  in 
the  unaided  condition.  The  ground  contact  data  showed  that  pilots  in  the  aided 
condition  were,  startlingly,  8  times  as  likely  to  inadvertently  contact  the  ground  as 
when  they  were  in  the  unaided  condition. 

Such  findings  are  reminiscent  of  the  series  of  studies  on  "attentional 
tunneling"  and  "cognitive  capture"  performed  over  the  past  15  years  (e.g., 

Weintraub,  Haines  &  Randle,  1984;  Larish  &  Wickins,  1991;  Foyle,  Sanford,  & 
McCann,  1991).  Indeed,  some  of  the  pilots  reported  a  kind  of  "fixation"  with  the 
waypoint  markers.  We  have  previously  analyzed  and  summarized  this  line  of 
research  (Rogers,  &  Spiker,  1995)  and  would  find  it  surprising  that  a  symbol  that  is  as 
uncomplicated,  conformal,  out-the-window-oriented,  and  holistically  processed  as 
the  waypoint  marker  could  lead  to  attentional  tunneling.  In  short,  it  should  be  little 
more  likely  to  induce  attentional  tunneling  than  a  recognizable  tree  at  the  waypoint. 

It  seemed  particularly  odd  that  the  experimenters  and  several  experienced  heli¬ 
copter  pilots  had  flown  with  this  symbology  in  our  laboratory  many  times  without 
experiencing  cognitive  capture  effects,  yet  every  one  of  the  pilots  in  the  experiment 
seemed  to  have  had  this  experience.  This  led  us  to  closely  re-examine  the  nature  of 
the  symbol  we  had  created.  There  are,  of  course,  many  variables  to  consider  in 
constructing  such  a  symbol  and  controlling  its  behavior.  Because  we  did  not  have 
the  specifications  for  the  Comanche  symbol,  we  simply  constructed  our  own.  It  now 
appears  that  one  of  our  design  decisions  may  have  created  an  illusion  that  confused 
the  pilots  in  the  experiment  especially  since  the  experiment  instructions  did  not 
include  certain  details  regarding  the  symbol's  behavior. 

Specifically,  although  the  waypoint  symbol  circle  grew  as  the  aircraft 
approached  the  waypoint,  the  perspective  could  not  be  perfectly  accurate.  For  exam¬ 
ple,  in  order  to  see  a  distant  waypoint  symbol  on  the  HMD,  it's  size  cannot  be 
allowed  to  diminish  too  much.  Also,  it's  size  cannot  increase  too  much  as  it  is 
approached  or  the  circle  will  become  too  large  to  be  seen  in  the  field  of  view  and  the 
staff  will  block  the  pilot's  view  of  the  terrain.  Thus,  we  set  distances  from  the  aircraft 
at  which  the  minimum  and  maximum  circle  sizes  would  be  reached. 

This  dynamic  scaling  method  presented  the  symbol  at  a  single  minimum  size 
when  5,000  feet  or  more  away  and  increased  it  proportionally  to  distance  until  it  was 
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1000  feet  away  (with  no  further  size  changes  at  lesser  distances).  At  1000  feet  or  less, 
the  symbol  appeared  to  be  about  105  feet  tall,  and  pilots  may  have  assumed  that  it 
was  similar  to  a  "highway  in  the  sky"  and  attempted  to  fly  under  or  through  the 
circle,  which  was  not  possible.  The  symbol  plane  was  always  kept  perpendicular  to 
the  viewer,  so  that  it  would  not  change  in  shape  with  changes  in  the  aircraft's 
position.  One  of  the  results  of  this  approach,  however,  is  that  the  symbol  would 
appear  to  "lay  down"  on  the  ground  as  it  was  overflown.  Thus,  attempting  to  fly 
though  the  circle  would  lead  directly  to  a  ground  impact. 

The  experimenters  and  other  pilots  who  had  previously  flown  the  system  were 
aware  of  the  symbol  behavior  and  thus  were  not  drawn  into  the  ground.  It  is 
probable  that  the  ground  impacts  observed  in  the  experiment  would  have  been 
eliminated  had  we  has  simply  told  the  subjects  to  "maintain  your  altitude  and  don't 
try  to  fly  through  the  hoop." 

Subsequent  to  the  completion  of  the  experiment  we  have  tested  a  "pitch 
enable"  behavior  for  the  waypoint  marker,  which  causes  the  symbol  to  realistically 
compress  as  it  is  overflown,  the  staff  becoming  shorter  and  the  circle  becoming 
increasingly  oval  and  then  disappearing.  This  feature  alone  seems  to  reduce  much 
of  the  perceptual  problem.  We  later  learned  that  the  Comanche  virtual  waypoint 
marker  circle  is  always  maintained  just  above  the  horizon,  partially  to  deter  such 
problems  (B.E.  Hamilton,  personal  communication,  September  23,  1998).  Of  course 
there  are  many  ways  to  provide  this  type  of  information  and  PRISMS  flexibility 
facilitates  the  testing  of  these  different  approaches.  For  example,  the  circle  could  be 
placed  at  a  fixed  altitude  above  ground  level  and  the  staff  extended  to  reach  the  way- 
point  on  the  ground,  so  that  a  concept  similar  to  the  highway-in-the-sky  could  be 
evaluated. 

There  remains  the  issue  of  the  greater  period  of  time  looking  forward  than 
rotating  the  head  to  scan  the  environment  when  the  waypoint  symbol  was  present. 
However,  several  pilot's  commented  that  the  fixation  on  the  symbols  was  experi¬ 
enced  only  because  they  were  new  and  novel  and  that  it  would  take  only  a  little 
time  to  become  accustomed  to  them.  One  pilot  said  that  proper  use  would  come 
with  experience  and  estimated  that  "another  hour  of  practice"  would  be  sufficient. 

Virtual  EA  Marker  Symbology 

The  extraordinary  success  of  the  EA  symbology  in  both  objective  and  subjective 
measures  has  very  important  operational  implications.  The  EA  is  typically  defined 
by  lines  drawn  on  a  paper  map  or  transparent  overlay  and  is  often  subdivided  into 
fire  sectors  assigned  to  different  friendly  units.  As  described  in  Field  Manual  1-112; 
Attack  Helicopter  Operations,  EAs  are  used  because  it  is  critically  important  to 
correctly  distribute  and  control  the  fires  from  available  weapons.  Attack  helicopters 
must  fire  only  in  their  assigned  sectors  in  order  to  prevent  fratricide,  to  avoid  target 
overkill  (such  as  firing  10  missiles  at  one  tank)  to  avoid  target  underkill,  and  to  use 
each  weapon  system  in  its  best  role. 
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Effective  use  of  EAs  depends  largely  upon  the  presence  of  some  easily  recogniz¬ 
able  features  in  the  terrain.  Unfortunately,  the  battlefield  area  does  not  always  offer 
obvious,  discernible  landforms  and  other  terrain  features  so  that  finding  the  EA  and 
the  correct  sector  at  night  in  unfamiliar  terrain  is  an  extremely  difficult  task.  Near 
the  battle  area,  the  CPG's  primary  responsibility  is  operating  the  aircraft's  complex 
weapon  systems,  and  he  has  little  time  to  aid  the  pilot  in  navigation  tasks. 

The  pilot  must  frequently  move  between  battle  positions  and  firing  positions 
in  order  to  deceive  the  enemy,  yet  orient  the  aircraft  properly  for  the  CPG's  use  of 
the  weapons  in  the  correct  firing  sector,  an  area  that  may  be  very  difficult  to  identify. 
The  problem  is  made  even  more  difficult  if  the  planned  mission  has  been  changed 
enroute,  due  to  unforeseen  changes  in  the  situation,  so  that  new  EA  positions  and 
sectors  must  be  assigned  by  radio.  By  comparison  to  actual  operational  conditions, 
the  unaided  experimental  task  was  relatively  easy.  As  demonstrated  by  the  PRISMS 
experiment,  however,  the  unmistakable  definition  of  fire  sectors  by  HMD  symbol- 
ogy  provides  dramatic  performance  advantages. 

Summary 

The  experiment  was  most  effective  in  demonstrating  the  overwhelming 
advantages  of  the  new  earth-fixed  symbol  types  for  use  with  HMDs  in  military  heli¬ 
copters.  The  accuracy  of  position-finding  in  the  terrain  improved  by  approximately 
300  to  400  percent  with  display  of  the  virtual  waypoint  symbols.  The  exposure  to 
enemy  forces  through  inadvertent  unmasking  was  reduced  by  approximately  one 
half.  The  EA  fire  sector  identification  accuracy  was  improved  by  about  12,000 
percent. 

In  addition  to  these  important  findings,  the  experiment  has  also  demonstrated 
the  relative  ease  with  which  PRISMS  can  be  used  to  construct  and  edit  experimental 
sessions,  add  and  improve  symbology  features  and  behaviors,  provide  realistic 
terrain  and  objects,  and  provide  an  extensive  range  of  performance  measures  --  all 
in  a  package  that  is  transportable  to  the  field.  PRISMS  has  not  only  fulfilled  its 
project  objectives,  but  will  continue  to  provide  researchers  with  a  powerful  but 
inexpensive  tool  for  many  years  to  come. 
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Section  7:  Symbology  Evaluations  with  PRISMS; 
Demonstrations,  Interviews,  and  Survey 


Introduction 

This  section  provides  a  summation  of  pilot  subjective  data,  amplifying  the 
objective  data  gathered  during  the  experiment  described  in  the  previous  section. 

The  first  two  subsections  present  the  pilots'  responses  to  the  waypoint  marker 
symbology  and  the  EA  marker  symbology.  Because  there  were  many  more 
opportunities  for  new  HMD  symbols  than  could  be  evaluated  in  formal  experiments 
in  the  scope  of  this  project  and  because  PRISMS  has  been  designed  for  "quick-look" 
evaluations  and  knowledge  acquisition  sessions,  we  elected  to  construct 
demonstration  sessions  to  show  the  pilots  some  of  these  new  concepts. 

The  five  new  concepts  demonstrated  included  special  new  symbology  for 
presenting  slope  landing  data,  wind  speed  and  direction,  required  speed  for  accurate 
arrival  time,  ASE  threat  direction,  and  flight  path  prediction.  Each  of  the 
demonstration  sessions  are  described  in  terms  of  the  background  and  reasons  for 
their  development,  the  instructions  given  to  the  pilots  before  the  demonstration, 
the  pilots'  overall  subjective  responses,  specific  topics  pertinent  to  each  symbol,  and 
the  pilots'  suggestions  for  intelligent  symbology  moding  opportunities. 

Although  the  experiment  and  the  five  demonstrations  permitted  the  gathering 
of  a  great  deal  of  valuable  data,  there  was  not  sufficient  time  for  even  cursory 
evaluations  of  the  scores  of  potential  symbols  identified  during  the  project.  In  order 
to  prioritize  future  research  directions  with  PRISMS,  we  conducted  a  survey  to 
determine  the  relative  importance  of  new  HMD  symbol  types.  This  final  section  of 
the  report  concludes  with  the  results  of  this  survey. 

Waypoint  Marker  Symbology  -  Summary  of  Pilots'  Comments 

Following  a  planned  route  at  low  nap-of-the-earth  (NOE)  flight  altitudes  is  an 
extremely  demanding  task.  Other  than  a  rough  sketch  map  on  his  kneeboard  and 
the  command  heading  caret  in  the  heading  scale,  the  Apache  pilot  is  almost  entirely 
dependent  on  the  copilot-gunner  (CPG)  for  navigation  information.  As  a  result,  the 
pilot's  situation  awareness  is  often  less  than  optimal.  It  is  usually  too  dangerous  to 
look  down  at  the  sketch  map  when  flying  NOE,  especially  at  night.  In  some  cases 
the  pilot  may  try  to  memorize  the  flight  legs  (the  heading  and  distance  to  each  new 
waypoint).  The  CPG  currently  must  help  the  pilot  by  "talking  him  through"  the 
waypoints  with  the  use  of  a  map  and  avionics  in  the  front  cockpit. 

Because  of  these  known  operational  difficulties  and  given  our  search  for  high-payoff 
improvements,  the  first  new  symbol  to  be  chosen  for  experimental  evaluation  was 
the  virtual  waypoint  marker,  similar  in  some  respects  to  one  of  the  advanced 
features  planned  for  the  RAH-66  Comanche  helicopter.  It  is  an  earth-fixed  symbol 
that  appears  to  the  pilot  to  be  a  giant  map-tack  or  "lollipop"  stuck  in  the  real-world 
terrain  to  show  upcoming  waypoints,  thus  reducing  navigational  workload. 
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Figure  7-1.  The  appearance  of  the  virtual  waypoint  marker  with  the  other  HMD 
symbology  (above)  and  an  explanation  of  the  meaning  of  the  dashed 
symbol  (below)  showing  a  waypoint  obscured  by  intervening  terrain. 


Instructions  to  Pilots  (Aided  Condition) 

The  pilots  were  instructed  to  examine  a  l:24,000-scale  paper  map  with  a  flight 
plan  overlay.  "The  mission  will  begin  just  northeast  of  Waypoint  1  and  continue 
through  Waypoints  2,  3,  and  4  to  a  Holding  Area,  then  on  to  a  Battle  Position.  Along 
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the  route  to  the  Holding  Area,  try  to  maintain  60  knots  and  100  ft  AGL  altitude.  You 
need  not  follow  straight  flight  legs  between  waypoints,  but  you  must  overfly  the 
waypoints.  In  addition  to  the  Command  Heading  Caret  under  the  magnetic  heading 
scale  at  the  top  of  the  display,  we  will  add  some  symbols  that  point  out  the  way- 
points  to  you.  They  look  like  "lollipops"  [The  SME  was  shown  a  drawing  of  the 
virtual  waypoint  symbol].  When  you  cross  a  waypoint,  the  symbol  will  appear  over 
the  next  waypoint.  If  the  next  waypoint  is  obscured  by  terrain  between  it  and  your 
position,  the  symbol  is  shown  by  a  red,  dashed  outline." 

In  addition,  another  figure  was  shown  to  the  pilots  to  depict  an  additional  cue 
to  waypoint  and  landing  positions.  The  cue  had  the  appearance  of  a  small  "igloo"  10 
feet  in  diameter.  This  symbol,  unlike  the  waypoint  symbol,  did  not  disappear  (jump 
to  the  next  waypoint)  as  the  aircraft  passed  the  waypoint  position.  Thus,  it  could  be 
used  as  a  continuing  reference  point  which  was  particularly  useful  for  landings. 

Overall  Response 

Without  exception,  the  pilots  praised  the  virtual  waypoint  marker  utility. 
Comments  included  "lots  of  merit,"  "definitely  a  workload  reduction,"  "I  love  it," 
"excellent,"  "very  useful,"  "boy,  I  like  that,"  "Nice,"  "definitely  going  to  help  you," 
and  "very  valuable."  They  noted  that  the  markers  were  much  better  than  following 
the  heading  cursor  because  it  was  unnecessary  to  fly  a  straight  line  between 
waypoints.  With  the  waypoint  markers,  the  pilot  could  follow  a  frequently 
changing  course  offering  the  best  cover  and  concealment  between  waypoints. 

During  planning,  the  pilots  attempt  to  select  easily  identifiable  features  like 
road  intersections  or  mountain  peaks  as  good  check  features,  but  such  features  are 
not  always  available,  or  may  be  difficult  to  see  in  the  actual  terrain.  As  another  pilot 
put  it,  the  "where  am  I?"  kinds  of  questions  "happen  constantly."  and  the  waypoint 
markers  would  "really  help  situation  awareness." 

They  observed  that  in  some  cases  the  pilot  doesn't  always  "have  tons  of  work  to 
do  if  the  guy  in  front  is  navigating,"  but  if  the  CPG  gets  "task-overloaded"  or  makes 
some  mistakes,  "things  become  much  tougher."  The  front-seater  is  particularly  busy 
working  on  navigation,  continuously  figuring  the  heading  and  distance  to 
waypoints,  using  a  strip  map  and  the  Doppler  system.  He  is  head-down  most  of  the 
time,  and  verbally  relays  course  and  speed  information  to  the  pilot,  attempting  to 
bring  the  aircraft  to  the  waypoints  accurately  and  on  time. 

Under  this  pressure  he  sometimes  makes  Doppler  input  mistakes  and  "the 
pilot  must  always  anticipate  where  his  directions  should  take  you"  and  tell  the  CPG 
when  he  must  be  wrong.  The  navigation  task  is  "especially  tough  if  you  are  the  lead 
aircraft  --  knowing  where  the  next  waypoint  is,  is  much  more  significant  with 
people  following  you." 

The  waypoint  markers  would  definitely  reduce  workload  for  both  men,  letting 
them  concentrate  on  other  tasks.  They  would  let  the  CPG  "hunt  for  targets"  and  let 
the  pilot  make  better  use  the  terrain  and  look  for  targets  of  opportunity. 
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Additional  Information  Recommended 

Nearly  all  of  the  pilots  believed  that  the  waypoint  marker  information  should 
be  augmented  to  include  waypoint  distance  from  the  aircraft,  in  kilometers.  As  one 
pilot  explained,  "The  waypoint  symbol  gives  an  intuitive  sense  of  direction,  but  not 
distance.  I  want  the  range  so  I  can  use  the  terrain  to  advantage."  The  range  of  the 
waypoint  is  currently  obtained  from  the  Doppler  by  the  CPG  and  passed  to  the  pilot. 
The  range  data  can  become  especially  important  when  trying  to  make  accurate 
passage  times,  "such  as  FLOT  passage  times." 

Several  pilots  suggested  that  the  waypoint  should  be  specifically  identified  by 
number.  Some  sort  of  coding  should  permit  easily  matching  a  waypoint  on  the  map 
to  the  waypoint  symbol  seen  in  the  HMD.  The  pilots  cautioned  that  the  Doppler 
numbers  are  not  the  same  as  the  waypoint  numbers.  For  example,  Doppler  Number 
5  could  be  Waypoint  Number  17,  which  tends  to  be  confusing.  One  pilot  said  he 
uses  the  taxonomy  of  "waypoint"  to  be  equivalent  of  Doppler  number,  and  "air 
control  point"  (ACP)  to  be  the  number  on  the  mission  plan  and  another  term  such 
as  FARP  to  describe  the  nature  of  the  position.  In  any  case,  the  waypoint  number  on 
the  mission  plan  is  the  one  that  should  be  shown  with  the  symbol. 

Some  of  the  pilots  also  thought  that  letters,  shape  codes,  or  color  codes  should 
be  used  to  identify  the  earth  fixed  symbols.  "You  could  use  T  for  a  target,  H  for 
holding  area,  F  for  FARP,  A  for  ACP,"  and  so  forth.  This  could  save  a  lot  of  time 
since  the  pilot  is  "always  asking  the  front  sea  ter  what's  where."  Another  code  might 
be  used  when  using  the  laser  to  "store  points  for  targets  that  the  aircraft  could  attack 
on  the  egress  route."  Using  conformal  markers  this  way  could  permit  designation 
of  many  more  targets  for  the  second  pass  through  the  same  route. 

The  pilots  were  divided  on  where  to  show  the  distance  and  waypoint  identifier. 
Most  believed  that  it  should  be  inside  the  waypoint  marker  circle,  or  adjacent  to  it, 
but  others  were  concerned  about  its  readability  if  small  enough  or  the  clutter  it 
would  introduce  if  large  enough.  Two  of  the  pilots  mentioned  that  their  unit  was 
about  to  receive  the  Embedded  GPS  Integration  (EGI)  retrofit.  When  a  waypoint  is 
selected,  this  unit  will  display  the  range  to  the  waypoint  in  the  right  side  of  the  High 
Action  Display  (HAD).  The  EGI  display  may  thus  provide  the  necessary  distance 
information  without  additional  clutter  to  the  center  of  the  waypoint  symbol. 

Moding  Control  Suggestions 

With  regard  to  symbol  moding,  the  automatic  change  to  show  the  next  way- 
point  as  each  waypoint  is  passed  was  met  with  approval.  All  of  the  pilots  stated  that 
they  wanted  at  least  one  (the  next)  waypoint  symbol  displayed  at  all  times.  Some 
said  that  they  would  like  to  be  able  to  call  up  any  other  waypoint  to  determine  its 
relative  location  and  distance.  Most  felt  that  a  single  waypoint  was  enough, 
although  a  second  might  be  useful  to  indicate  the  direction  of  the  next  turn.  One 
pilot  suggested  that  if  the  WAS  switch  was  used  to  select  Hellfire  or  rockets,  the 
waypoint  symbols  could  be  removed  to  reduce  clutter  and  the  command  heading 
caret  used  instead. 
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Engagement  Area  Symbology  -  Summary  of  Pilots'  Comments 

Background 

This  new  symbol  belongs  to  a  class  of  more  complex,  earth-fixed  markers 
("augmented  reality")  proposed  after  our  interviews  with  expert  Apache  pilots 
during  the  Phase  I  effort  This  class  of  symbols  would  aid  the  pilot  by  pointing  out 
critical  tactical  demarcations  and  zones  on  the  ground  such  as  phase  lines,  planned 
artillery  fires,  international  or  unit  boundaries,  holding  areas,  battle  positions,  and 
target  engagement  areas.  Such  markers  are  presently  defined  on  transparent  opera¬ 
tions  overlays  superimposed  on  paper  topographic  maps  that  are  very  difficult  to 
use  in  the  cockpit.  If  they  were  instead  presented  with  HMD  symbology  in  an  earth- 
fixed  frame  of  reference  so  as  to  appear  "painted"  on  the  ground,  the  benefits  could 
include  reduced  navigational  workload,  improved  tactical  situation  awareness,  and 
diminished  risk  of  fratricide  (destruction  of  friendly  forces). 

Instructions  to  Pilots  (Aided  Condition) 

When  the  pilot  had  reached  the  BP,  he  was  directed  to  look  downward  to  view 
the  map  and  its  tactical  overlay  data  showing  the  EA.  "As  you  can  see  on  the  map, 
the  EA  is  divided  into  three  firing  sectors.  Your  team  has  been  assigned  the  center 
sector.  On  my  command,  you  will  bob  up  to  an  altitude  sufficient  to  view  the  EA 
sectors  drawn  out  on  the  terrain  and  observe  a  column  of  enemy  tanks.  You  are  to 
fire  on  only  two  tanks:  the  one  closest  to  the  left  boundary  of  the  center  sector  and 
the  one  closest  to  the  right  boundary  of  the  center  sector.  Both  shots  must  be 
completed  within  15  seconds  after  the  bob-up,  then  bob  down  to  land  at  the  BP." 

Overall  Response 

The  pilots  were  perhaps  even  more  enthusiastic  about  the  EA  symbology  than 
the  waypoint  symbols.  Comments  included  "huge  utility,"  "worth  their  weight  in 
gold,"  "enormously  easier,"  "I  love  that  symbol,"  "awesome,"  "so  much  easier,"  and 
"tremendous  workload  reduction." 

All  of  the  pilots  confirmed  the  difficulty  of  the  fire  control  and  distribution 
process.  "Units  spend  a  lot  of  valuable  time  at  the  BP  trying  to  get  this  right."  Pilots 
can't  tell  where  the  targets  are  without  good  target  reference  points,  and  these  are 
rarely  available.  Currently,  in  the  operational  setting,  "dicing  up  the  EA  is  always  a 
challenge."  Even  though  there  may  be  roads  and  streams  in  the  area,  actually  seeing 
them  at  night  "can  be  quite  challenging."  The  EA  symbol  would  "save  time, 
ordnance,  confusion,  and  money,"  and  avoid  the  problems  of  fratricide  and  "hitting 
the  same  target  many  times."  The  EA  symbol  would  lead  to  "a  marked  increase  in 
the  economy  of  ammunition  used."  Without  these  symbols,  "it's  a  stab  in  the 
dark."  None  of  the  pilots  characterized  the  experimental  task  as  unrealistically  diffi¬ 
cult. 


7-5 


Information  Distribution 

Two  pilots  observed  that  it's  important  that  both  the  pilot  and  the  CPG  have 
the  EA  symbols  for  situation  awareness  and  coordination.  A  very  experienced  pilot 
stated  that  the  fire  control  sectors  in  the  EA  should  be  identified  during  mission 
planning,  using  the  AMPS,  and  this  information  should  somehow  be  directly 
routed  to  the  HMD  symbology  generator.  The  same  pilot  noted,  however,  that 
sometimes  when  the  unit  arrives  at  the  BP,  the  tactical  situation  has  shifted  and  in 
such  cases  it  would  be  extremely  valuable  for  the  battalion  commander  to  be  able  to 
"data-burst"  new  fire  sectors  to  the  companies.  Two  pilots  suggested  that  "you 
should  show  battalion,  company,  and  fire  team  sectors,"  and  some  suggested  the  use 
of  color  codes  for  identifying  the  various  sectors. 

Use  as  a  Masking  Cue 

The  use  of  the  dashed  EA  lines  as  a  cue  to  masking  from  the  enemy  positions 
received  mixed  reviews.  Many  found  the  broken  red  line  masking  cue  to  be  very 
valuable.  One  pilot,  for  example,  stated  that  "the  use  of  the  dotted  lines  as  you're 
moving  in  to  get  to  the  BP  helps  a  lot."  Another  said  "I  can't  believe  how  much 
better  I  flew  this"  (section  of  the  route)  because  of  the  dashed  line  cue  to  masking. 
Some  who  praised  the  solid  lines  nevertheless  found  the  dashed  lines  "potentially 
confusing"  because  of  their  unfamiliar  perspective.  The  confusion  might  disappear 
with  more  exposure  to  the  dashed  line  symbol.  As  one  pilot  said,  "with  more  expe¬ 
rience"  he  would  "learn  to  use  the  solid  and  broken  lines  more  effectively." 

Moding  Control  Suggestions 

Pilots  offered  a  number  of  potential  strategies  for  when  the  EA  symbology 
should  appear.  Several  felt  it  should  be  enabled  after  the  HA  or  after  the  Release 
Point  (the  point  of  dispersion  of  company  aircraft  after  the  HA  and  before  the  indi¬ 
vidual  BPs). 

Some  felt  that  the  symbol  could  come  on  at  some  standard  range,  such  as  "15 
kilometers,"  "5  kilometers,"  "3  to  6  kilometers,"  or  "no  later  than  3  to  4  kilometers." 
But  others  said  that  the  range  would  depend  upon  the  type  of  terrain  in  the  area  and 
the  airspeed. 

One  of  the  pilots  indicated  the  EAs  should  be  displayed  in  accordance  with  the 
"weapons  and  vulnerability"  situation,  particularly  the  "maximum  effective 
ranges."  For  example,  "if  the  BPs  put  you  outside  the  enemy  engagement  range, 
then  it  would  not  be  necessary  to  put  the  EA  up  sooner  than  the  maximum  effective 
range"  of  one's  own  weapons.  Thus,  if  in  premission  planning,  the  maximum 
ADA  range  was  known,  "you  could  use  that"  (to  control  EA  appearance)  "and  fly 
with  a  little  more  confidence."  using  the  EA  symbol  absence  as  a  "safe  zone"  indica¬ 
tor. 

Several  pilots  stated  that  they  would  like  to  be  able  to  use  a  manual  control  or  a 
voice  command  to  call  up  the  EA  symbology  whenever  they  wanted  it,  and  just  as 
quickly,  to  get  rid  of  the  symbology  and  its  potential  visual  clutter. 
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Slope  Landing  Symbology  Demonstration 

Background 

Although  pilots  would  always  prefer  to  land  on  flat  terrain,  sometimes  landing 
on  a  slope  is  unavoidable.  The  Apache  slope  landing  limits  are  10°  of  roll,  7°  nose- 
up,  and  12°  nose-down.  Unfortunately,  no  clinometer  is  provided  in  the  aircraft  for 
determining  the  steepness  of  slopes.  For  example,  with  a  lateral  slope,  the  pilot 
contacts  the  slope  with  the  upper  wheel,  then  gently  lowers  the  downhill  wheel 
toward  the  ground  by  lowering  collective,  while  attempting  to  judge  whether  the 
limit  is  about  to  be  exceeded  from  the  trim  ball  and  out-the-window  visual  cues. 

Instructions  to  Pilots 

"Apache  pilots  know  that  the  slope  landing  limits  for  the  AH-64  are  10°  of  roll,  7° 
nose-up,  and  12°  nose-down.  However,  judging  the  actual  slope  of  terrain  is  difficult.  The 
next  symbol  we  want  to  show  you  is  the  Slope  Landing  Aid,  or  inclinometer,  [the  pilot  was 
shown  an  illustration  as  the  symbology  function  was  explained). 

When  one  wheel  touches  the  ground,  four  tic  marks  appear  in  the  HMD.  The  two 
marks  in  the  center  are  used  with  the  horizon  line  to  show  nose-up  and  nose-down  limits. 
The  two  tics  toward  the  right  sight  show  the  left  and  right  roll  limits. 

We'll  start  by  looking  at  the  symbology  in  an  aircraft  that  has  already  landed  on  a 
fairly  steep  hillside.  If  you  do  a  slow  pedal  turn  on  the  ground  you  can  see  how  the 
horizon  line  changes  with  respect  to  the  tic  marks  as  the  direction  of  slope  changes. 

Now  try  gently  picking  the  aircraft  up  and  landing  to  see  how  the  symbols 
change  and  go  on  and  off.  As  you  lower  the  collective,  you  can  watch  the  horizon 
line  and  the  tick  marks  to  see  if  the  slope  is  acceptable  for  a  landing." 

Overall  Response 

Five  Apache  pilots  evaluated  the  Slope  Landing  Aid,  and  all  agreed  that  it  would  be 
a  very  useful  addition.  Additional  comments  included  "1  like  it,"  "It's  easy  to 
understand,"  "It  would  be  good  for  any  kind  of  landing,"  "very  helpful,"  and  "I  can 
work  with  this  easy." 

Typical  Slope  Landings 

The  pilots  stated  that  it  is  very  hard  to  tell  how  steep  the  slope  is,  especially  at  night. 

As  a  result,  they  currently  avoid  slope  landings  at  night  unless  they  are  absolutely 
required.  In  addition,  at  night  a  landing  spot  may  appear  to  be  flat  when  in  fact  it  is 
not.  After  initial  touchdown,  there  is  about  6  inches  of  compression  of  the  strut, 
then  some  expected  additional  compression  of  the  snow  or  dirt.  Then  surprises  may 
happen.  Day  or  night,  slight  slopes  can  suddenly  become  much  greater  when  a 
wheel  slips  over  a  ledge,  sinks  into  a  gopher  hole,  or  settles  deeper  into  the  snow. 


7-7 


Figure  7-2.  The  four  tic  marks  used  with  the  horizon  line  for  slope  landing. 
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To  counter  these  surprises,  Apache  pilots  carefully  "feel"  their  way  down,  to 
make  sure  they  do  not  exceed  the  angular  limits  of  10°  left  or  right  roll,  7°  nose-up, 
and  12°  nose  down.  Currently,  their  only  instrumentation  cue  for  slope  landings  is 
the  trim  ball.  The  trim  ball  "rolls"  along  the  top  of  the  field  of  regard  box,  and  when 
it  reaches  the  point  at  which  it  is  centered  on  the  edge  of  the  box,  the  aircraft  is 
approximately  at  a  10°  roll  angle.  The  ball  stops  at  the  edge  of  the  box,  so  pilots  can't 
actually  tell  if  the  10°  roll  angle  has  been  exceeded.  Of  course,  the  trim  ball  is  not  a 
useful  cue  for  nose-up  or  nose-down  landings,  and  it's  "especially  hard  to  tell  for 
nose-up  and  nose-down  conditions"  whether  the  slope  limits  are  being  exceeded. 

Moding  Control  Suggestions 

Pilots  agreed  that  the  use  of  the  wheel  touch-down  would  be  a  good  way  to 
make  the  slope  landing  symbol  appear.  One  noted  that  "it's  good  to  know  that 
event  anyway;"  that  is,  the  symbol  appearance  would  be  useful  in  indicating  a 
touchdown.  "You  can't  trust  the  altitude  sensor  for  indicating  your  touchdown." 

Unfortunately,  the  Apache  "squat  switch"  is  currently  implemented  only  on 
the  left  side  of  the  aircraft.  This  could  cause  a  problem  when  aircraft  on  the  ground 
are  aligned  in  "herringbone  or  wagon-wheel  patterns."  Two  of  the  pilots  suggested 
that  a  second  squat  switch  could  probably  be  added  at  minimal  cost.  Another 
approach  would  be  to  use  an  accurate  "ground  proximity  switch."  If  a  ground  prox¬ 
imity  approach  were  to  be  available,  "you  could  simply  use  the  last  four  feet  of  alti¬ 
tude"  (to  turn  on  the  symbol)  because  below  that  is  the  "limit  of  hovering  flight,"  an 
Army  standard. 

Three  of  the  pilots  pointed  out  that  in  addition  to  use  during  slope  landings,  it 
was  "nice  to  land  and  get  the  horizon  line"  without  having  to  go  back  to  the  transi¬ 
tion  mode  from  the  hover  mode  through  actuating  the  thumb  switch. 

One  pilot  pointed  out  that  the  symbol  would  not  have  to  be  visible  whenever 
the  aircraft  was  on  the  ground,  but  would  only  be  needed  upon  landing,  and  not  at 
takeoff.  So,  "if  the  aircraft  systems  were  turned  on  while  the  aircraft  was  on  the 
ground,  the  tic  marks  and  horizon  line  would  not  be  there,  but  would  appear  just 
on  landing,"  an  easy  rule  to  implement. 

One  pilot  suggested  making  the  tic  marks  appear  a  little  bit  brighter  or  a  differ¬ 
ent  color  than  the  horizon  line  for  ease  of  interpretation. 

Wind  Indicator  Cue  Symbology  Demonstration 

Background 

The  Apache  airspeed  symbol  in  the  HMD  does  not  currently  show  the  relative 
direction  of  the  wind,  although  it  does  show  a  speed  even  if  the  helicopter  is  not 
moving.  During  Phase  I,  Apache  pilots  told  us  that  wind  velocity  and  direction  data 
could  be  helpful  in  certain  situations  such  as  "cranking  or  shutting  down,  takeoff 
with  a  tail  wind,  or  descending  into  trees  with  a  tail  wind  that  might  push  you  into 
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trouble."  In  addition,  wind  data  could  be  valuable  for  "rocket  engagements,"  "hover 
taxi,"  "restricted  area  landings,"  "bob-ups,"  "landing  at  unfamiliar  fields,"  and 
"hovering  at  night  in  the  trees."  Head  winds  and  tail  winds  are  the  most  difficult  to 
detect,  since  no  aircraft  attitude  changes  are  available  as  a  cue.  Knowledge  of  wind 
direction  can  also  be  important  for  aircraft  recovery  maneuvers,  such  as  after  loss  of 
an  engine,  when  flying  into  the  wind  may  be  particularly  desirable. 


Instructions  to  Pilots 

"The  next  symbol  we  want  to  show  you  is  a  wind  indicator.  As  you  know,  the  Apache 
true  airspeed  readout  does  not  show  the  direction  of  the  wind,  although  it  does  show  a 
speed  even  if  the  helicopter  is  not  moving  over  the  ground.  We've  been  told  that  a 
symbol  showing  the  wind  speed  and  direction  relative  to  the  aircraft  could  be  useful  (The 
pilot  was  shown  a  figure  illustrating  the  wind  symbol). 

This  symbol,  shown  in  the  lower  left  comer  of  the  display,  shows  the  windspeed 
numerically.  It  uses  a  rounded  pointer  to  show  the  direction  of  the  wind  with  respect  to 
the  aircraft.  This  illustration  shows  how  the  symbol  would  indicate  a  15  knot  wind  from 
about  7  o'clock,  (the  pilot  then  donned  the  helmet). 

First,  orient  the  aircraft  to  the  north  and  do  a  bob-up  maneuver  to  about  10  feet,  (a  15 
knot  wind  from  270°  was  detected  as  the  aircraft  left  the  ground). 
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As  you  see,  there  is  a  wind  active  in  the  area.  Touch  down  and  we'll  turn  on  the 
wind  indicator  symbol.  Now  you  can  bob-up  again  knowing  which  way  and  how  hard  the 
wind  is  blowing." 

Overall  Response 

All  of  the  five  pilots  who  tried  the  wind  indicator  cue  found  it  to  be  "useful"  or 
"very  useful."  Other  comments  included  "immediately  understandable,"  "I  like  it," 
"would  be  handy,"  "would  save  you  time,"  and  "it  can  only  help." 

Application  with  Weapon  Engagements 

The  pilots  offered  a  range  of  applications  for  the  wind  symbol  based  on  their 
operational  requirements.  Three  of  the  pilots  noted  that  wind  direction  "is  impor¬ 
tant  for  rocket  shots."  There  is  a  "zero-knot  limit  for  tail  winds  for  firing  of  rockets, 
and  there's  another  limit  for  Hellfire  missiles,  so  this  would  be  useful  for  weapon 
engagements."  One  of  the  pilots  stated  that  "the  rocket  ADS  ballistic  solutions  do,  in 
fact,  include  wind  calculations;  however  they  are  not  shown  to  the  pilot."  He  also 
observed  that  even  though  it's  undesirable,  "sometimes  you  still  have  to  shoot  with 
the  wind  up  your  tail."  When  firing  rockets,  knowing  the  winds  would  also  "be 
handy  for  using  Kentucky  windage." 

Application  in  Mountain  Operations 

Two  of  the  pilots  stressed  that  in  mountain  operations  "it's  very  important  to 
understand  the  winds  for  pinnacle  or  ridge  line  operations."  These  pilots  "do  a  lot  of 
power  management  work  at  6,000'+  altitudes,  up  to  7500',  so  winds  are  critical  in 
hovering  and  landing."  Pilots  must  be  sure  to  fly  into  the  wind  for  pinnacle 
approaches.  They  currently  perform  a  "high  recon,"  flying  a  circular  pattern  around 
a  position  to  determine  the  direction  and  strength  of  the  winds.  The  wind  symbol 
"would  save  a  lot  of  workload  in  determining  the  best  approach,"  and,  "since  at 
airspeeds  of  greater  than  35  knots  you  don't  make  such  landings,  it  would  be  nice  to 
know  that  figure,  as  well." 

Applications  in  Other  Operational  Requirements 

The  pilots  described  a  number  of  other  situations  in  which  the  wind  indicator 
cue  would  be  valuable.  "If  you're  shooting  an  approach,  you  want  to  see  if  there's  a 
tail  wind.  It's  hard  to  tell  unless  you're  very  experienced."  "When  you  perform 
health  indicator  checks  for  the  engines,  you  must  orient  the  aircraft  into  the  wind." 

"It  would  also  be  useful  for  roll  on  landings  to  unimproved  areas.  That  way  you 
could  both  put  the  aircraft  into  the  wind,  and  by  flying  into  the  wind,  keep  the  dust 
behind  you  when  you  land."  The  symbol  would  be  useful  "particularly  on  cranking 
or  shutting  down."  "It  would  also  be  very  useful  if  you  had  a  single  engine  out." 

"You  want  to  land  into  the  wind  when  you're  at  the  FARP,  so  it  would  be  useful  for 
that."  "It  would  enhance  situation  awareness  such  as  hovering  at  night  near  trees." 

"It  is  also  useful  for  showing  which  way  the  dust  and  snow  will  blow."  "A  lot  of 
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times  I  wish  I  knew  what  the  windspeed  was,  especially  on  take-off  from  the  FARP 
or  the  holding  area.  I  currently  use  dust  to  figure  out  which  way  the  wind  is 
blowing." 

Symbol  Appearance  and  Placement 

None  of  the  pilots  took  issue  with  the  symbol  appearance.  For  example,  one 
said  "This  symbol  is  immediately  understandable  without  any  explanation.  It  ap¬ 
pears  like  a  wind  sock."  All  indicated  that  the  symbol  location  was  acceptable 
although  one  pilot  noted  that  "you  may  want  to  raise  it  up  a  bit  to  make  sure  it 
doesn't  interfere  with  the  lower-left  side  of  the  high  action  display." 

Moding  Control  Suggestions 

Two  of  the  pilots  thought  that  a  manual  moding  technique  or  a  voice 
command  would  work  best  to  bring  up  the  symbol  or  delete  it,  whenever  desired. 
Looking  "down  into  the  cockpit"  should  definitely  not  be  required  to  set  this  mode. 
Others  believed  that  certain  combinations  of  airspeed,  windspeed,  wind  direction, 
and  altitude  should  be  used  to  call  up  the  symbol  and  thereby  alert  the  pilot.  One 
said  that  "it  should  be  visible  when  the  winds  are  above  10  knots  in  any  direction." 
Another  pilot  suggested  that  "below  6  knots  means  wind  speed  equals  light  and 
variable,  so  perhaps  below  6  knots  wind  speed  you  would  not  need  to  display  it." 
Another  stated  that  it  should  be  "made  visible  from  take-off  through  20  knots  above 
effective  translational  lift." 

One  of  the  pilots  pointed  out  that  "usually  only  winds  in  excess  of  30  knots  are 
any  kind  of  problem  for  the  Apache,  although  a  tail  wind  of  5  knots  could  be  impor¬ 
tant."  Another  stated  that  "you  would  not  need  the  indicator  at  higher  altitudes 
enroute  and  in  the  cruise  mode."  Yet  another  said  "you  could  probably  take  it  off 
when  you're  above  5  knots  airspeed."  The  pilots  that  discussed  the  use  of  the 
symbol  in  rocket  operations  pointed  out  that  moding  would  be  easy;  "whenever  you 
use  the  WAS  switch  to  select  rockets,  you  should  get  the  wind  symbol  coming  up." 

Speed-to-Fly  Symbology  Demonstration 

Background 

Accuracy  in  arrival  times  at  certain  waypoints  or  other  tactical  positions  can  be 
critical  to  mission  success  and  aircraft  survival.  For  example,  inbound  and 
outbound  Passage  Point  arrival  times  must  be  very  accurate  (±  30  seconds  accuracy  is 
allowable)  to  avoid  risking  "friendly  fire."  The  existing  airspeed  display  is,  by  defini¬ 
tion,  influenced  by  winds  and  does  not  provide  ground  speed  unless  the  pilot  "fails" 
the  ADS.  As  an  alternative,  the  CPG  can  be  further  burdened  with  a  request  to 
repeatedly  calculate  GS  and  tell  the  pilot  to  "speed  up  or  slow  down  a  little."  Given 
the  current  speed,  the  Doppler  system  alone  can  be  used  to  determine  the  ETA  at  a 
waypoint.  However,  the  system  does  not  "know"  whether  the  aircraft  is  on  time,  or 
by  how  much  it  is  ahead  or  behind  schedule.  Some  kind  of  cue  to  determining 


7-12 


correct  time  of  arrival  could  be  a  significant  workload  reducer  for  both  the  pilot  and 
the  CPG. 


Instructions  to  Pilots 

"The  next  new  symbol  we  want  to  show  you  is  called  the  Speed-to-Fly  Cue.  Pilots 
have  told  us  that  hitting  certain  waypoints  on  time  can  be  important,  such  as  at  passage 
points.  The  Doppler  system  can  calculate  ground  speed  and  ETA  at  a  waypoint,  but  does 
not  'know'  if  the  aircraft  is  ahead  or  behind  schedule. 

We  can  compare  the  ETA  with  the  required  waypoint  time  and  display  a  symbol  that 
looks  like  this  next  to  the  regular  airspeed  indicator  (the  pilot  was  shown  an  illustration). 

The  'up'  or  'down'  direction  of  the  little  arrow  next  to  the  airspeed  tells  the  pilot 
whether  to  increase  or  decrease  airspeed,  and  the  number  to  the  right  how  much  to  change 
it  for  accurate  arrival  time  at  the  next  waypoint.  We'll  begin  near  Waypoint  1  and  fly  to 
the  HA." 

Overall  Response 

All  of  the  five  pilots  that  tried  this  symbol  found  it  to  be  very  useful.  Other 
comments  included  "definitely  a  useful  addition,"  "much  faster,"  "more  accurate," 
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"could  relate  to  time  easily/'  "would  reduce  workload,"  and  "its  location  and 
appearance  are  just  perfect." 

Improvement  Over  Current  Operations 

The  specific  waypoint  times  derive  from  the  planning  process.  "You  back-plan 
from  the  FLOT  time  or  other  critical  position,  and  then  set  the  times  for  each  point, 
given  the  ground  speed  and  the  distance."  Pilots  usually  "fudge  for  the  winds,"  and 
do  a  Doppler  check  along  the  way.  Enroute,  the  pilot  "can  turn  the  ADS  (air  data 
sensor)  off  and  get  the  ground  speed.  But  most  pilots  prefer  to  leave  the  ADS  on  for 
use  with  waypoints. 

The  pilots  were  adamant  regarding  the  necessity  of  accurate  waypoint  arrival 
times.  One  said  that  "Timing  is  really  important,  especially  in  and  out  of  the 
passage  point,  the  cross  FLOT  times  must  be  within  plus  or  minus  30  seconds.  It's 
critical."  Another  stated  that  waypoint  times  are  "nearly  always  important,  but  are 
critical  with  SEAD  (suppression  of  enemy  air  defense)  missions  when  there  are  artil¬ 
lery  barrage  times"  to  be  considered.  "To  be  either  early  or  late  over  these  points  is 
dangerous." 

For  accurate  data,  the  pilot  has  to  ask  the  CPG  for  time-distance  information. 
The  CPG  must  use  the  Doppler  (which  provides  ground  speed  in  knots)  and  do  "a 
lot  of  calculations."  He  determines  the  necessary  true  airspeed  for  the  aircraft  to 
deliver  the  proper  ground  speed  for  each  leg,  then  relays  the  information  to  the 
pilot,  suggesting  that  he  increase  or  decrease  current  airspeed  to  make  the  waypoint 
time. 

The  pilots  noted  that  with  the  speed-to-fly  symbol,  all  this  calculation  and 
communication  would  be  unnecessary.  "It  would  also  prevent  a  lot  of  speed 
fluctuations  between  waypoints."  Every  one  of  the  pilots  considered  the  speed-to-fly 
cue  to  be  a  very  useful  addition. 

One  pilot  observed  that  it  would  be  possible  to  use  the  speed-to-fly  cue  as  an 
indicant  of  "when  to  transition  to  the  start  point  on  the  egress  route."  For  example, 
if  the  crew  had  15  minutes  station  time  plus  2  minutes  to  get  to  the  start  point,  "in 
your  setup  you  could  enter  17  minutes  to  the  start  point  from  the  BP  arrival  time." 
Then  "you  would  just  wait  until  the  display  says  it's  time  to  leave."  That  is,  the 
symbol  would  appear  with  an  up  arrow  and  a  speed  increase  (such  as  "80  knots"). 

Or,  if  the  aircraft  left  the  BP  late  for  some  reason,  "it  would  give  you  the  needed 
speed  to  make  that  start  point  at  the  egress  route." 

All  agreed  that  the  location  and  its  format  are  just  fine  as  they  are.  One  pilot 
suggested  color  coding  the  number  or  putting  the  number  above  or  below  the 
airspeed  to  show  the  direction  of  change  so  that  the  arrow  would  be  unnecessary 
and  some  clutter  could  be  saved. 
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Moding  Control  Suggestions 

One  of  the  pilots  thought  "it  would  be  great  to  have  on  continuously,"  and 
another  recommended  a  manual  on-and-off  switch  for  cases  in  which  they  don't 
require  this  information  during  flight.  "The  switch  could  be  on  the  cyclic  or  a  voice 
command  could  be  used." 

One  pilot  suggested  that  the  display  of  the  speed-to-fly  cue  "could  start  at  a  40- 
knot  airspeed  or  perhaps  20."  The  important  concept  is  to  recognize  when  the 
aircraft  is  actually  flying,  not  hovering.  For  example,  "we're  doing  120  knots  at  the 
release  points."  When  the  aircraft  is  hovering  and  "moving  short  distances  such  as 
around  the  BP,"  the  speed-to-fly  cue  would  "not  be  much  help."  Another  agreed 
that  "it  could  probably  be  decluttered  in  some  NOE  mode"  since  "100-120  knots  is  a 
typical  enroute  ground  speed,  and  the  passage  points  are  generally  overflown  at  50 
feet." 

Pilots  were  also  asked  how  great  an  error  from  the  ideal  flight  airspeed  should 
trigger  the  appearance  of  the  arrow  and  number.  The  demonstration  used  a  1  knot 
difference.  Several  pilots  agreed  that  1  knot  was  appropriate,  although  one  pilot 
suggested  a  5-knot  error  as  the  cue  for  the  symbol  to  appear.  Another  insightfully 
suggested  that  during  the  use  of  "your  mission  planning  software,  you  could  deter¬ 
mine  the  different  boundaries  for  when  the  cue  would  appear,  perhaps  down  to  1 
knot  differences  when  you  are  close  to  the  BP,"  or  some  other  critical  position. 

ASE  Threat  Cue  Symbology  Demonstration 

Background 

Although  the  APR-39  radar  warning  device  in  the  AH-64A  provides  voice 
warnings  of  threat  activity  and  their  relative  direction,  such  as  "Searching  -  2 
o'clock,"  Apache  pilots  interviewed  during  Phase  I  had  indicated  that  HMD 
symbology  might  provide  a  better  situation  awareness  cue.  In  order  to  permit  rapid 
masking  of  the  aircraft,  instead  of  receiving  just  a  numerical  bearing  to  the  threat,  it 
was  suggested  that  the  HMD  present  an  enemy  weapon  symbol  in  the  field  of  regard, 
perhaps  supplemented  with  a  3-D  audio  cue,  to  more  naturalistically  represent  the 
weapon's  position  in  space. 

Since  the  APR-39  does  not  provide  distance-to-weapon  information,  the 
symbol  could  not  be  used  to  identify  a  specific  position  on  the  ground,  but  would 
have  to  show  the  azimuth  to  the  weapon.  Pilots  have  indicated  that  the  HMD 
symbology  could  give  a  spatially  superior  indicant  of  direction  with  a  line  or  some 
other  marker  so  that  the  pilot  could  either  orient  the  aircraft  appropriately  for  use  of 
the  gun,  jammer,  chaff,  or  flares,  or  prepare  to  deploy  to  cover.  Unlike  a  spoken 
warning,  a  visible  line  provides  a  continuous  indicant  of  the  direction  of  the  threat, 
even  as  the  pilot  changes  his  heading  to  respond  to  the  threat. 
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Instructions  to  Pilots 

“The  next  symbol  is  an  improvement  for  the  APR-39.  In  addition  to  the  voice 
warning  of  the  rough  direction  of  threat  activity,  an  ASE  Threat  Cue  Symbol  could  be 
presented  in  the  HMD,  to  point  out  the  direction  of  the  threat.  The  symbol  we  will  show 
you  is  a  pair  of  vertical  lines.  The  target  will  be  found  between  them.  This  symbol  is  used 
if  no  good  target  range  data  is  available. 

[A  figure  depicting  the  symbol  was  shown  to  the  pilots]  Let’s  fly  through  part  of  the 
waypoints  from  the  first  experiment  and  see  how  the  ASE  Threat  Cue  Symbol  would  point 
out  the  threat  positions.  A  3D  sound  cue  will  help  to  indicate  the  direction  of  the  threat. 
Fire  on  the  targets  then  continue  route  at  low  altitude.” 


Figure  7-5.  The  ASE  Threat  Cue  Symbol  indicating  a  target  azimuth. 


Overall  Response 

All  five  of  the  pilots  who  tried  the  ASE  Threat  Cue  symbol  were  very  favorably 
impressed.  Comment  included  “I  like  it  a  lot,”  “this  would  be  great,”  “pretty  neat,” 
“absolutely  a  useful  addition,”  “this  is  a  novel,  good  thought,”  and  “it  sure  beats 
looking  down  at  that  stupid  thing”  (the  APR-39).  It  is  noteworthy  that  during  their 
previous  two  flights  through  the  same  area  these  five  pilots  detected  and  fired  upon 
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an  average  of  1.2  targets  per  flight.  With  the  ASE  Threat  Cue  symbol,  of  course,  all 
four  of  the  targets  were  easily  found. 

Improvement  Over  Current  Operations 

The  current  APR-39  gives  a  verbal  message  such  as  "ZSU  11  o-clock  tracking" 
and  presents  "a  quick  strobe  on  the  head-down  display."  But  "the  strobe  disappears 
right  away  and  you  may  miss  it  entirely."  The  pilot  doesn't  always  see  the  APR-39 
strobe  line  and  "a  clock  position  is  useless  if  you're  maneuvering  too  late  to  figure 
out  the  real  direction  of  the  strobe."  In  addition,  "this  information  must  be  relayed 
from  the  pilot  to  the  front  seater."  The  front  seater  "has  to  ask  the  back  seat  where 
the  strobes  are  coming  from  so  that  he  can  effectively  slew  the  weapons."  Alterna¬ 
tively,  "he  can  slew  his  TADS  line  of  sight  to  the  helmet  line  of  sight,"  but  using  the 
new  ASE  Threat  Cue  symbol  is  much  faster,  assuming  both  the  front  and  the  back 
seat  have  it.  One  pilot  said  that  "using  it  for  air-to-air  would  be  great,  too." 

Use  of  3D  Sound 

The  3D  sound  is  "great'  because  it  "tells  you  where  to  look  immediately." 
Otherwise,  "it's  hard  to  figure  out  with  the  current  APR-39."  Some  pilots  felt  that  a 
simple  buzzer  sound  such  as  used  in  the  demonstration  would  be  adequate  because 
"knowing  it's  there  is  more  important  than  getting  an  idea  of  the  specific  threat." 
Two  pilots  thought  the  buzzer  sounded  too  much  like  "an  engine-out  warning." 

Other  pilots  pointed  out  that  the  current  APR  39  uses  a  voice  message  to  indi¬ 
cate  the  type  of  weapon  of  the  threat  and  that  the  type  of  weapon  is  still  important 
information,  suggesting  that  the  auditory  portion  of  the  cue  should  perhaps  use  the 
name  of  the  weapon  instead  of  a  simple  tone.  "A  woman's  voice  would  be  okay." 
The  other  words  of  the  APR-39  message,  such  as  "tracking,  acquisition,  launch,  or 
lock  broken,"  should  also  be  included,  but  done  in  3D  sound. 

Two  pilots  suggested  that  a  "sound-off  button"  or  a  voice  command  to  shut  up 
the  warning  noise  would  be  good,  until  another  new  one  came  on.  "You  might 
need  some  kind  of  back-up,  such  as  a  4-minute  reminder,  so  that  the  warning  would 
come  back  on  if  you'd  been  ignoring  it." 

One  pilot  warned  that  there  is  a  problem  to  consider  in  using  the  3D  auditor 
warnings  for  pilots.  "Pilots  have  a  lot  of  time  around  turbine  engines  and  many 
have  had  hearing  problems."  Thus,  especially  with  lots  of  noise,  "localization  is  a 
problem  and  you  can't  always  depend  on  that  as  a  cue."  Redundant  cues  such  as  a 
pointer  or  compass  heading  information  to  the  threat  might  be  useful  for  crew 
coordination. 

Symbol  Appearance 

Most  of  the  pilots  indicated  that  the  symbol  appearance  was  "good,"  or 
"appropriate,"  and  were  not  particularly  concerned  about  the  clutter  the  two  lines 
would  add  at  this  point  in  the  mission.  As  one  of  the  pilots  put  it,.  "Don't  worry 
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about  having  two  lines— clutter  is  not  a  big  issue  in  this  case.  When  the  target  comes 
up,  it's  the  most  important  thing."  Two  of  the  pilots  liked  the  two-line  approach, 
but  suggested  that  they  be  in  a  color  unlike  that  of  the  other  symbology  or  the  target 
itself. 

Target  Prioritization 

Several  of  the  pilots  suggested  some  form  of  additional  information  be  added, 
for  example,  something  akin  to  that  currently  used  with  the  voice  warnings  of  the 
APR-39.  One  pilot  suggested  that  it  might  be  good  to  prioritize  pairs  of  lines  for 
multiple  targets  by  using  shape  codes  for  our  lines.  "You  might  want  to  be  able  to 
discriminate  between  tracking  or  lock  or  launch  so  that  the  first  priority  would  be 
launch,  the  second  lock,  the  third  tracking,  with  some  kind  of  shape  code."  he 
suggested  dotted  lines  for  tracking  and  solid  for  acquisition,  "or  color  codes  if  they 
are  possible."  He  also  noted  that  the  APR-39  currently  shows  small  symbols  indicat¬ 
ing  the  type  of  enemy  weapon,  so  these  could  be  considered  for  the  head-up  applica¬ 
tion  as  well. 

One  pilot  observed  that  friendlies  might  be  in  the  area  and  "you  should  use  IFF 
codes  to  provide  blue  symbols  for  the  friendlies  for  fratricide  prevention  versus  red 
for  the  opposing  forces."  There  was  also  some  questions  regarding  the  best  way  to 
show  multiple  enemy  weapons.  It  might  be  possible  to  draw  lines  part  way  down 
from  the  top  or  part  way  up  from  the  bottom  to  show  secondary  priority  targets, 
although  either  way  could  begin  to  introduce  clutter. 

Moding  Control  Suggestions. 

Pilots  agreed  that  the  events  that  would  be  detected  to  make  the  symbol  come  on 
would  be  the  same  kind  of  sensor  events  that  the  APR-39  currently  uses. 

Flight  Path  Marker  Symbology  Demonstration 

Background 

The  flight  path  marker  symbol  shows  the  continuously  computed  velocity  vector  of 
the  aircraft.  The  IHADSS  symbology,  of  course,  includes  a  velocity  vector  that  is  a  "top- 
down"  view  useful  for  hover  control.  The  flight  path  marker  symbol,  however,  is  an 
"out-the-window"  view  along  the  axis  of  the  velocity  vector,  showing  where  the  aircraft 
will  fly  or  contact  the  ground  if  no  changes  are  made  to  the  controls.  Thus,  the  most 
obvious  virtue  of  the  flight  path  marker  symbol  is  for  avoidance  of  controlled  flight  into 
terrain  (CFIT)  accidents.  The  knowledge  of  the  aircraft's  velocity  vector  can  be  useful  in 
many  other  ways  as  well,  including  terrain  following,  turn  coordination,  and  precision 
landings.  Although  the  flight  path  marker  symbol  was  not  derived  from  our  information 
requirements  analyses,  its  experimental  use  during  PRISMS  NOE  flights  through  rugged 
terrain  made  its  virtues  so  dramatically  evident  that  we  chose  to  include  it  in  the 
demonstration  sessions. 
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Figure  7-6.  Appearance  of  the  flight  path  marker  with  the  other  HMD  symbols 
(upper)  and  example  of  the  flight  path  marker  changing  size  (lower). 

Instructions  to  Pilots 

"The  next  symbol  we  want  to  show  you  is  called  the  Flight  Path  Marker.  It  shows  the 
continuously  computed  velocity  vector  of  the  aircraft.  You  can  use  it  to  see  exactly  where 
the  aircraft  will  fly  or  contact  the  ground  if  control  inputs  do  not  change.  We  have  it  set 
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up  so  that  the  symbol  "grows"  in  size  as  the  impact  point  becomes  nearer  to  the  aircraft, 
and  will  begin  flashing  at  3  seconds  to  impact. 

First,  let's  compare  landing  at  a  specific  position  without  and  with  the  Flight  Path 
Marker.  From  about  500  feet,  land  at  the  pad.  Next,  we'll  try  it  with  the  Flight  Path  Marker 
on.  Just  keep  the  Flight  Path  Marker  positioned  on  your  desired  landing  point. 

(Later,  after  landing  exercises) 

Now  let's  try  it  out  first  flying  low  and  fast.  Observe  how  it  moves  with  your  control 
inputs  and  how  it  changes  size.  See  if  you  can  get  the  flashing  warning  of  3  seconds  to 
ground  impact." 

Overall  Response 

All  of  the  five  pilots  who  evaluated  the  flight  path  marker  symbol  were 
strongly  in  favor  of  its  addition  to  the  symbol  set.  Comments  included  "a  really 
good  addition,"  "it  would  be  great,"  "very  useful,"  "excellent,"  "pretty  nice,"  "it's  a 
good  tool,"  "especially  useful  at  night,"  "a  precise  power  manager,"  and  "it  could 
save  your  life." 

Utility  in  Tactical  Flight 

All  of  the  pilots  agreed  that  "this  would  be  a  really  good  addition  for  low-level 
flight,"  especially  in  for  terrain  following  and  terrain  avoidance."  Accurately 
clearing  ridgelines  without  "ballooning  to  400  feet"  is  another  requirement  that 
would  be  aided  by  the  flight  path  marker.  One  pilot  observed  that  "it's  good  for 
night  flight,  especially,  because  it's  hard  to  see  the  terrain  then."  Another  pilot  tried 
it  with  aggressive  turns  and  pronounced  it  "great  for  sharp  turns  while  you're  losing 
altitude." 

Utility  for  Landings 

Pilots  also  agreed  that  "it's  good  for  setting  up  landings,"  because  it's  not  neces¬ 
sary  to  "look  inside  the  cockpit  for  information"  during  the  landing.  One  stated  that 
it  was  "great  for  shooting  approaches"  because  the  pilot  doesn't  "have  to  check 
several  different  things."  Another  said  "I  like  it.  Especially  for  roll  on  landings."  It 
provides  a  "good  rate  of  descent  cue,"  and  the  landing  position  can  be  fine- timed, 
using  the  symbol  as  a  "precise  power  manager,"  keeping  the  symbol  over  the 
intended  landing  spot.  It  would  also  be  useful  for  autorotations,  setting  the  aircraft 
attitude  to  produce  an  80-knot  airspeed  and  putting  the  symbol  on  a  desired  landing 
area. 

Utility  with  Weapons 

One  pilot  was  particularly  interested  in  the  use  of  the  flight  path  marker  with 
rocket  fire,  suggesting  that  the  symbol  might  indicate  "both  the  aircraft's  and  the 
rocket's  point  of  impact  in  the  terrain."  when  using  direct  fire  methods.  Another 
pilot  noted  that  the  symbol  would  be  valuable  in  "determining  the  pull-up  point  for 
diving  fire"  with  the  gun  or  rockets. 
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Utility  in  Flight  Instruction 

An  instructor-pilot  indicated  that  as  a  safety  cue,  flashing  the  flight  path  marker 
symbol  would  "be  excellent  for  flight  with  students.  At  low  level,  high  speed,  you 
really  need  to  detect  descent  in  a  hurry."  He  and  another  pilot  suggested  that  an 
auditory  warning  message  such  as  "Pull  up"  be  added  to  correspond  with  the 
flashing  flight  path  marker. 

Symbol  Implementation 

Pilots  were  very  much  in  favor  of  the  flashing  symbol  indicating  imminent 
ground  impact,  one  saying  "it  could  be  what  saves  you,"  and  another  saying  "it  will 
help  you  survive  in  flight."  However,  there  was  some  disagreement  over  the 
warning  period.  The  demonstration  used  a  three-second  warning,  but  three  of  the 
pilots  suggested  that  5  seconds,  might  provide  a  greater  margin  of  safety.  The  size¬ 
changing  feature  was  judged  to  "get  your  attention  well."  One  pilot  suggested  that 
in  addition  to  flashing,  perhaps  it  "ought  to  change  to  red."  Another  suggested  that 
it  be  made  to  look  more  different  from  the  acceleration  cue,  to  avoid  any  confusion. 

Moding  Control  Suggestions 

One  of  the  pilots  suggested  that  the  flight  path  marker  be  continuously 
displayed  for  safety  purposes  and  as  a  training  aid  for  improving  landing  technique. 

Four  of  the  pilots  stated  that  it  could  be  intelligently  moded  in  accordance  with  alti¬ 
tude  and  airspeed.  One  said  that  "it  should  be  on  at  all  times  when  less  than  500  feet 
AGL."  Another  said  "above  200  feet  you  would  not  need  this  symbol,"  and  yet 
another  said  it  should  appear  at  "less  than  100  feet."  One  suggestion  was  that  the 
flight  path  marker  "would  be  unnecessary  in  hover  or  bob-up  modes,"  and  could  be 
turned  off  in  those  cases,  but  should  come  back  on  when  airspeed  reached  "30  to  40 
knots."  Another  pilot  said  that  the  symbol  should  be  shown  for  NOE  flight,  or 
"anytime  below  80  knots  and  below  100  feet." 

One  of  the  pilots  described  the  NOE  approach  switch  setting,  and  its  effect  on 
the  flight  path  marker.  This  switch  "schedules"  (moves)  the  stabilator  so  that  the 
aircraft  nose  comes  down  for  better  visibility.  That  is,  it  "moves  the  nose  down 
without  moving  the  rotor  disk."  Thus,  if  this  switch  were  actuated,  "you  might  not 
want  to  display  the  flight  path  marker,"  or  at  least  a  correction  to  the  symbol  loca¬ 
tions  should  be  studied. 

Survey  of  Symbol  Usefulness 

Although  a  large  number  of  potential  HMD  symbols  had  been  identified  in  our 
analyses  (as  described  in  Section  4),  only  a  few  examples  of  various  symbol  types  could  be 
evaluated  with  the  PRISMS  simulator  in  the  time  available  with  the  Apache  SMEs.  In 
fact,  the  PRISMS  simulator  was  developed  for  years  of  future  use  based  on  recognition  of 
the  overwhelming  number  of  demonstrations,  evaluations,  and  experiments  that  should 
be  undertaken  in  specifying  the  content  and  characteristics  of  new  HMD  symbology. 
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Development  of  the  Survey  Tool 

Having  identified  so  many  potential  symbols,  we  were  anxious  to  prioritize  them  in 
terms  of  their  probable  payoff  for  Army  pilots.  Having  just  completed  a  series  of  symbol 
evaluations  on  the  PRISMS  simulator,  it  seemed  that  the  pilots  would  be  ideally  prepared 
for  judging  the  utility  of  other  new  symbols.  Thus,  a  survey  and  rating  scale  approach 
appeared  to  be  the  most  appropriate  method  for  creating  such  a  prioritization.  Unfortu¬ 
nately,  the  number  of  information  elements  in  our  full  list  would  have  resulted  in  a 
survey  that  would  have  been  quite  long  and  tedious  for  the  pilots.  Another  problem  with 
attempting  such  ratings  was  that  many  of  the  new  symbols  would  be  difficult  to  under¬ 
stand  without  a  detailed  explanation,  further  increasing  the  demands  of  a  survey  task 
beyond  acceptable  limits. 

In  order  to  reduce  the  severity  of  both  problems,  we  elected  to  review  our  list  of 
potential  symbols  and  reduce  their  numbers  by  selecting  only  those  that  would  easily  be 
understood  by  experienced  pilots  without  special  explanations.  As  a  result,  the  final  list  of 
survey  items  consisted  of  85  information  elements.  The  survey  was  presented  to  the  pilots 
after  they  had  completed  90  minutes  of  PRISMS  symbology  evaluations  with  a  request  that 
they  complete  and  return  it  to  Anacapa  Sciences  as  soon  as  possible.  The  survey  instruc¬ 
tions  were  as  follows: 

Please  consider  your  answers  to  the  following  questions  carefully.  Your 
responses  will  help  to  guide  the  next  generation  of  HMD  symbology.  All 
aviators'  answers  will  be  kept  strictly  confidential  by  Anacapa  Sciences. 

During  the  demonstrations  in  the  PRISMS  simulator,  you  saw  several  new 
kinds  of  HMD  symbols.  Having  had  this  introduction,  you  can  more  easily 
imagine  how  other  new  symbols  can  be  created.  Some  might  be  words  or 
numbers,  and  others  might  be  "earth-fixed"  and  point  out  positions  or  areas 
in  the  real  world,  like  the  waypoint  markers  and  EA  boundaries.  If  the  right 
rules  can  be  developed,  these  new  symbols  could  be  made  to  appear  when 
needed,  then  disappear  so  as  not  to  clutter  the  pilot's  view. 

Listed  below  are  a  series  of  information  elements  that  have  been  suggested  for 
presentation  as  symbols  on  the  HMD.  Please  rate  their  usefulness  as  an  HMD 
symbol,  assuming  that  the  information  element  was  presented  only  when 
needed. 


A  7-point  rating  scale  was  used  in  which  "1"  was  equivalent  to  "Not  useful," 
and  "7"  was  equivalent  to  "Very  useful,"  as  shown  in  the  sample  below: 

(Circle  the  appropriate  number) 

Information  Element 

Not  Very 

Useful  Useful 

1.  Wind  direction  relative  to  the  aircraft  1....2....3....4....5....6....7 
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As  a  further  confirmation  of  the  most  critical  new  symbols,  once  the  SMEs  had 
finished  the  ratings  they  were  instructed  to  look  back  through  their  ratings  and 
identify  the  20  most  important  symbols.  They  were  further  instructed  that  "You 
may  have  to  make  some  additional  decisions  about  which  ones  are  most  important 
if  some  are  tied  at  the  same  rating  number." 

Results  of  the  Survey 

Six  of  the  study  participants  completed  the  entire  survey  form  and  returned  it 
to  us.  The  results  are  based  upon  these  six  sets  of  ratings  and  are  shown  in  Table  7.1 
rank-ordered  by  importance  of  the  information  element.  The  number  to  the  left  of 
the  information  element  is  its  position  in  the  list  of  85  symbols.  The  number  to  the 
right  of  the  information  element  is  the  mean  rating  score. 

Table  7.1 

Information  Elements  Ranked  by  Usefulness 

74.  Airspace  zones:  threat  lethality  envelopes  6.67 

79.  Enemy  forces  locations  6.67 

12.  Distance  remaining  to  next  waypoint  6.50 

44.  ASE  warning:  threat  weapon  status  6.50 

47.  Ground  positions:  engagement  areas,  fire  sectors  6.50 

48.  Ground  positions:  next  waypoint  6.33 

54.  Ground  positions:  battle  positions  6.33 

58.  Ground  positions:  FARPs  6.33 

60.  Ground  positions:  friendly  troops  6.33 

42.  ASE  warning:  head-up  direction  cue  to  threat  6.17 

43.  ASE  warning:  head-up  position  cue  to  threat  6.17 

50.  Ground  positions:  passage  points  6.17 

51.  Ground  positions:  air  control  points  6.17 

52.  Ground  positions:  rally  points  6.17 

53.  Ground  positions:  holding  areas,  assembly  areas  6.17 

61.  Ground  positions:  friendly  artillery  6.17 

7.  Ground  speed  required  for  on-time  arrival  at  waypoint  6.00 

9.  Time  remaining  for  on-time  arrival  at  waypoint  5.83 

49.  Ground  positions:  any  selected  waypoint  5.83 

56.  Ground  positions:  no-fire  zones  (NFA)  5.83 

59.  Ground  positions:  phase  line  and  force  boundaries  5.83 

68.  Airspace  positions:  flight  team  members  5.83 

1.  Wind  direction  relative  to  the  aircraft  5.67 

62.  Airspace  positions:  air  targets  5.67 

81.  Forward  Line  of  Own  Troops  (FLOT)  5.67 

83.  Landing  Zones  (LZ)  5.67 

10.  Time  remaining  for  on-time  arrival  at  destination  5.50 

57.  Ground  positions:  fire  support  coordination  lines  5.50 

82.  Hazards  to  flight  5.40 

3.  Current  ground  speed  (knots)  5.33 
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Table  7.1.  Continued 

Information  Elements  Ranked  by  Usefulness 


8.  Airspeed  required  for  on-time  arrival  at  waypoint  5.33 

67.  Airspace  positions:  air  targets  detected  by  sensor  5.33 

80.  Forward  Edge  of  the  Battle  Area  (FEBA)  5.33 

2.  Wind  speed  relative  to  the  aircraft  5.17 

35.  Flight  path  marker  (as  shown  in  PRISMS)  5.17 

45.  Time  of  guided  missile  flight  for  LOAL  5.17 

46.  Time  to  designate  target  for  LOAL  5.17 

55.  Ground  positions:  free-fire  areas  (FFA)  5.17 

33.  Bearing  pointer  to  ADF  or  VOR  5.00 

76.  Speed-to-fly  cue,  like  shown  with  PRISMS  5.00 

84.  Fire  support:  Restrictive  Fire  Line  (RFL)  5.00 

63.  Airspace  positions:  next  to  shoot  target  4.83 

17.  Autorotation  flare  timing  and  pitch  cue  4.67 

36.  Numeric  heading  indicator  4.67 

11.  Mission  time  clock  4.50 

15.  Roll-on  landing  warning:  parking  brake  on  4.50 

16.  Roll-on  landing  warning:  tail  wheel  unlocked  4.50 

65.  Airspace  positions:  restricted  areas  4.50 

77.  Slope  landing  aid,  like  shown  with  PRISMS  4.50 

73.  Airspace  zones:  NBC  contamination  4.40 

30.  Distance  and  direction  to  planned  track  4.33 

69.  Airspace  zones:  air  routes  and  corridors  4.33 

85.  Coordinated  Fire  Line  (CFL)  4.33 

37.  Synthetic  canopy  rails  for  night  pilotage  4.25 

4.  Current  ground  speed  (kilometers /hour)  4.17 

5.  Planned  ground  speed  (knots)  4.17 

32.  Course  deviation  bar  4.17 

20.  Alert:  Fire  or  other  caution  or  warning  4.00 

21.  Radio  frequency  numerics  (when  setting)  4.00 

22.  Transponder  numerics  (when  setting)  4.00 

31.  Current  ground  track  4.00 

6.  Planned  ground  speed  (kilometers /hour)  3.83 

38.  Minimum  single  engine  airspeed  3.83 

70.  Airspace  zones:  approach  holding  patterns  3.83 

18.  Alert:  Minimum  enroute  altitude  3.67 

19.  Alert:  Minimum  obstruction  clearance  altitude  3.67 

66.  Airspace  positions:  rendezvous  aircraft  3.67 

71.  Airspace  zones:  approach  path  3.67 

72.  Airspace  zones:  missed  approach  path  3.67 

78.  Control  setting  warning:  (e.g.,  DASE  off)  3.67 

75.  Training  cues  like  shown  with  PRISMS  pirouette  3.60 

13.  Current  barometric  altitude  (MSL)  3.50 

28.  Fuel  remaining  (time  at  present  burn  rate)  3.50 
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Table  7.1.  Continued 

Information  Elements  Ranked  by  Usefulness 


34. 

True  course 

3.50 

14. 

Planned  barometric  altitude  (MSL) 

3.33 

24. 

Pitch  angle  scale  (ladder) 

3.33 

41. 

Obstacle  avoidance  distance  cues 

3.20 

39. 

Time  until  damage  from  high  torque  condition 

3.17 

25. 

Roll  angle  scale 

3.00 

26. 

Engine  temperature 

3.00 

27. 

Fuel  remaining  (pounds) 

3.00 

40. 

Obstacle  avoidance  directional  cues 

3.00 

23. 

Radio  call  sign 

2.67 

29. 

Rotor  speed  indicator 

2.67 

64. 

Airspace  positions:  refueling  tanker 

1.50 

The  results  of  the  second  task,  identifying  the  twenty  most  important  informa¬ 
tion  elements  was  included  to  supplement  the  7-point  scaling  method  in  case  the 
ratings  were  concentrated  in  one  sector  of  the  scale.  The  results  of  this  method  are 
shown  in  Table  7.2.  It  is  interesting  to  note  that,  in  general,  there  was  good  corre¬ 
spondence  between  the  rating  scale  method  and  the  top-twenty  method.  Further¬ 
more,  there  was  quite  good  agreement  among  pilots  with  regard  to  the  top  twenty; 
all  pilots  agreed  that  the  most  important  information  element  was  "Enemy  forces 
locations,"  and  at  least  half  of  the  respondents  agreed  on  the  top  twenty  items. 

The  importance  of  symbols  indicating  tactical  ground  positions  is  powerfully 
confirmed  by  both  survey  approaches.  In  the  rating  scale  technique,  15  out  of  the 
most  important  21  features  were  ground  positions  (including  enemy  forces  loca¬ 
tions,  and  in  the  top-twenty  method,  9  of  the  20  most  important  symbols  identified 
ground  positions. 

It  is  noteworthy  that  of  the  20  most  important  symbols,  10  had  been  previously 
selected  for  use  in  our  PRISMS  experiments  and  demonstrations.  Although  it  is 
possible  that  a  case  could  be  made  for  the  PRISMS  sessions  having  influenced  subse¬ 
quent  ratings,  we  believe  that  this  finding  simply  reflects  the  accuracy  of  our  esti¬ 
mates  of  high-payoff  symbols  based  upon  the  Phase  I  tasks  as  well  as  many  years  of 
experience  in  analysis  of  Army  aviation  mission  requirements. 

In  any  case,  the  survey  data  presented  in  this  section  should  provide  valuable 
guidance  for  HMD  symbology  research  in  the  near  future.  Although  the  informa¬ 
tion  content  for  each  symbol  has  been  identified,  each  symbol's  format  and  behavior 
must  be  determined  through  further  demonstrations  and  evaluations.  It  is  our 
hope  that  the  PRISMS  simulator  will  play  a  key  role  in  the  performance  of  this 
research  for  years  to  come. 
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Table  7.2. 

The  Twenty  Most  Important  Information  Elements  as  Reported  by  the 
Survey  Respondents  (percentage  of  respondents  shown  at  left). 


100% 

79. 

Enemy  forces  locations 

67% 

7. 

Ground  speed  required  for  on-time  arrival  at  waypoint 

67% 

8. 

Airspeed  required  for  on-time  arrival  at  waypoint 

67% 

42. 

ASE  warning:  head-up  direction  cue  to  threat 

67% 

44. 

ASE  warning:  threat  weapon  status 

67% 

48. 

Ground  positions:  next  waypoint 

67% 

50. 

Ground  positions:  passage  points 

67% 

51. 

Ground  positions:  air  control  points 

67% 

54. 

Ground  positions:  battle  positions 

67% 

58. 

Ground  positions:  FARPs 

67% 

60. 

Ground  positions:  friendly  troops 

67% 

74. 

Airspace  zones:  threat  lethality  envelopes 

50% 

1. 

Wind  direction  relative  to  the  aircraft 

50% 

9. 

Time  remaining  for  on-time  arrival  at  waypoint 

50% 

12. 

Distance  remaining  to  next  waypoint 

50% 

35. 

Flight  path  marker  (as  shown  in  PRISMS) 

50% 

43. 

ASE  warning:  head-up  position  cue  to  threat 

50% 

47. 

Ground  positions:  engagement  areas,  fire  sectors 

50% 

49. 

Ground  positions:  any  selected  waypoint 

50% 

67. 

Airspace  positions:  air  targets  detected  by  sensor 
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